Bates Digital Music Symposium 2018

socketMusic: wireless (2015)

for improvised percussion, electronic sound, and mobile devices

by Derek Kwan



  • general info (form/compositional details/aesthetics)
  • breakdown tech stack
    • use of chance procedures as improv aid
    • technical considerations
  • further explorations

Background Info


  • (improvised) percussion: snare drum (+ found metal), hi hat
  • electronics: processed samples, processed percussion, synthesized sound
  • mobile devices: sounds triggered by web server


  • three large sections with contrasting textures + improv. styles
    • drone + hiss / modem
      • in part inspired by Contact opening scene (zoomout)
      • textural improv on SD/HH with dreadlock brushes
      • cell phones accumulate in radio samples / filtered noise bursts
      • chopped/reversed input recordings

Form cont.

  • (three large sections cont.)
    • radio + static
      • based around 60 bpm pulse
        • delayed beeps + second roving beep at 59.7015 bpm
      • hihat rolls synced with radio/static
        • multi-tap delay with each delay randomly panned
      • cell phones imitate modem sound from end of sec 1 (enter sequentially)

Form cont.

  • (three large sections cont.)
    • telegraph
      • hiss + clicks on speakers
      • sea of morse code pulses (cell phones) imitated on HH/metal on SD
      • ends with brown noise swell
    • Note: all durations and subevents determined at runtime in Pure Data


  • working at Stony Brook's Teaching Learning Lab (media lab) - wanted to apply computer programming to doctoral recital
  • inspiration - Steven Schick premiere - Lei Liang's Trans (2014) (Miller Theater, Columbia)
    • distributed stones to audience, signaled when to strike from stage
  • wanted an inexpensive way to do multichannel audio


  • wireless - in content and medium
  • breaking cultural (concert hall) norms
    • performer/composer/audience relationship/balance of power
      • improvisation, chance procedures on server/patch/mobile devices
      • audience participation (active vs passive)
    • stage/hall invisible wall
    • performer/stage as focus of attention - audio/visuals on cell phone
    • not silencing cell phones

Aesthetics cont.

  • spatialization
    • massively multichannel
    • immersivity (sound sources from audience)
    • chance-based location and volume (depends on audience)

Aesthetics cont.

  • randomization
    • hands-free organicity (division of duties)
    • computer as performer (living piece)
    • unpredictability (vs fixed-media/performer+tape)
    • "fuzzy"/aleatoric composing
      • composing in general outlines/proportions, leaving computer to fill out details
      • specifying rates of triggering, chances of triggering, ranges of parameters


  • majority of sounds reference wireless technology (samples, synthesis)
    • wireless telegraph (radio waves) - Guglielmo Marconi (patented 1896)
    • first public radio broadcast - Lee De Forest/Metropolitan Opera (12/3/1910)
      • Enrico Caruso as Canio - I Pagliacci (Leoncavallo)
      • Riccardo Martin as Turridu - Cavalieria Rusticana (Mascagni)

Sounds cont.

  • 1200 baud modem (not quite wireless, but evokes the WiFi-enabled nature of the piece)
  • other misc sounds (radio static, fake talk radio)

Technical Details

Software Tech Stack

  • Pure Data - brains
    • sound for on-stage speakers, scheduling, cues
  • Node.js (Macchiato/Clojurescript) - messasge forwarder
    • host web page
    • OSC communication with Pure Data
    • WebSocket communication with individual clients (
      • triggered events from Pd via OSC

Software Tech Stack cont.

  • HTML/CSS/JS(CLJS) - message implementer
    • client-facing view
    • React/Reagent/JS - reactive visuals/info display
    • Web Audio API/Tone.js - synths, samples
    • Websocket communication with server via

Hardware Tech Stack

  • laptop
  • audio interface (+ mics)
  • wireless router

Pure Data


  • audio for on-stage speakers
    • live input recording (attack-triggered at max rate of 20s) / processing (rev./chopped, delays, FFT bin mag. thresholding)
    • samples + synthesis
  • user interface (controls, clocks, cues)
    • clocks for overall piece, current section, current section duration, next event timing
    • sec. 2 - cues for roll entrances 8th note before start and finish

Jobs cont.

  • scheduling and triggering
    • determine structure of piece (3 major sections)
    • communication with node.js server (OSC)
      • retrieve client list (base64)
      • direct server to trigger events (WebSockets)

Chance Operations

  • overall section durations
    • randomly chosen within ranges to add up to at least seven minutes
  • timings of events/subsections
    • approx. times with given ranges (smw-nextsubsec 50 7 -> 50 seconds in given section +/- 7)
    • perc. part cued as well! (sec. 2)

Chance Operations cont.

  • specific contents of events
    • random selection of given range of clients
    • random selection and playback location of sound files (sec. 2)
    • randomization of parameters of synthesized sounds (sec.1 drone)
    • random triggering (and selection of params within given ranges) of samples
    • randomization of delay lines (section 2)
  • all based around abstractions of [random] object (LCG)

Node.js server


  • serve web page for clients
  • keep track of indiv. connections via WebSockets (
    • trigger events on all devices (broadcast)
    • trigger events on a specific device (emit)
  • communicate with Pure Data (OSC)
    • provide list of connections (base64)
    • forward triggers from Pd to clients

Client (HTML/CSS/JS via CLJS)


  • client-facing interface loaded on devices
    • CSS, JS for dynamic background, React (Reagent) for dynamic text
  • sound synthesis/playback (Web Audio API / Tone.js)
    • triggered by Node webserver (with visual feedback)

Chance Operations

  • all sequences generated client-side (audio.cljs)
    • chopped samples (sec. 1), filtered noise (sec. 1), telegraph beeps (sec. 3)
    • freq / dur / Q / sample location / amp. / etc.
  • start times of sequences (client.cljs)
    • flags from OSC for immed. start / delayed
  • other sound parameters
    • base freq. for modem synths (clicks/whine)
    • speed of modem clicks (subdivisions of 4s)
    • freq of sine osc. for telegraph

Technical Discussion

Technical Notes

  • why browser and not native?
    • easily (comparatively) cross-platform
    • no need to download additional software
    • server/client
    • cons
      • data payload size
      • limited to browser capabilities/non-native

Technical Considerations

  • accessing hosted web site?
    • static IP (easy, /etc/network/interfaces on *nix)
    • router and DNS server?
      • OpenWrt (DNS)
      • Dynamic DNS (host on computer with bind?)
    • qr-code direct to static ip
  • port number (port 80/custom port)
    • port 80 (Web): setcap 'cap_net_bind_service+3p' /usr/bin/nodejs
qr-code for static-ip addr. w/ page served on 8080
gen. w/ Racket Simple-Qr library (Chen Xiao)

Technical Considerations cont.

  • Apple/iOS - need user interaction to start audio context
  • latencies from WiFi (rhythmic/timing accuracy)
  • auto-screen lock without interaction
    • NoSleep.js -

Further Exploration

  • not only for concert hall (installations)
  • streaming live sound - maybe Icecast/WebRTC/recording-loading sounds?
  • more sophisticated visualizations
    • P5.js, Canvas, Three.js, Quil, AR.js...
  • greater user interactivity
    • many options for input (text, accelerometers, sensors, GUI)
    • user control of sound, user supply of sound
    • interactions: user-server, user-user
      • user data


  • "[Met Performance] CID:49210." MetOpera Database: The Metropolitan Opera Archives, Metropolitan Opera Association, Archive of December 3, 1910 Matinee Program Information

Bibliography cont.

  • Kane, Joseph Nathan. Famous First Facts. The H.W. Wilson Company, 1964.
  • Fantel, Hans. "Sound; Out of De Forest And Onto The Air Came Music." The New York Times, 14 Jan. 1990, p. 002030,

Bibliography cont.

  • Sims, Michael. "Cavalleria Rusticana and I Pagliacci Crimes of Passion." Cavalleria Rusticana and I Pagliacci,
  • Guglielmo Marconi - Biographical. Nobel Media AB 2018. Sun. 14 Oct 2018.