Bates Digital Music Symposium 2018
socketMusic: wireless (2015)
for improvised percussion, electronic sound, and mobile devices
by Derek Kwan
http(s)://derekxkwan.com
https://derekxkwan.github.io/smwireless-slides/
Outline
general info (form/compositional details/aesthetics)
breakdown tech stack
use of chance procedures as improv aid
technical considerations
further explorations
Instrumentation
(improvised) percussion: snare drum (+ found metal), hi hat
electronics: processed samples, processed percussion, synthesized sound
mobile devices: sounds triggered by web server
Form
three large sections with contrasting textures + improv. styles
drone + hiss / modem
in part inspired by Contact opening scene (zoomout)
textural improv on SD/HH with dreadlock brushes
cell phones accumulate in radio samples / filtered noise bursts
chopped/reversed input recordings
Form cont.
(three large sections cont.)
radio + static
based around 60 bpm pulse
delayed beeps + second roving beep at 59.7015 bpm
hihat rolls synced with radio/static
multi-tap delay with each delay randomly panned
cell phones imitate modem sound from end of sec 1 (enter sequentially)
Form cont.
(three large sections cont.)
telegraph
hiss + clicks on speakers
sea of morse code pulses (cell phones) imitated on HH/metal on SD
ends with brown noise swell
Note: all durations and subevents determined at runtime in Pure Data
Background
working at Stony Brook's Teaching Learning Lab (media lab) - wanted to apply computer programming to doctoral recital
inspiration - Steven Schick premiere - Lei Liang's Trans (2014) (Miller Theater, Columbia)
distributed stones to audience, signaled when to strike from stage
wanted an inexpensive way to do multichannel audio
Aesthetics
wireless - in content and medium
breaking cultural (concert hall) norms
performer/composer/audience relationship/balance of power
improvisation, chance procedures on server/patch/mobile devices
audience participation (active vs passive)
stage/hall invisible wall
performer/stage as focus of attention - audio/visuals on cell phone
not silencing cell phones
Aesthetics cont.
spatialization
massively multichannel
immersivity (sound sources from audience)
chance-based location and volume (depends on audience)
Aesthetics cont.
randomization
hands-free organicity (division of duties)
computer as performer (living piece)
unpredictability (vs fixed-media/performer+tape)
"fuzzy"/aleatoric composing
composing in general outlines/proportions, leaving computer to fill out details
specifying rates of triggering, chances of triggering, ranges of parameters
Sounds
majority of sounds reference wireless technology (samples, synthesis)
wireless telegraph (radio waves) - Guglielmo Marconi (patented 1896)
first public radio broadcast - Lee De Forest/Metropolitan Opera (12/3/1910)
Enrico Caruso as Canio - I Pagliacci (Leoncavallo)
Riccardo Martin as Turridu - Cavalieria Rusticana (Mascagni)
Sounds cont.
1200 baud modem (not quite wireless, but evokes the WiFi-enabled nature of the piece)
other misc sounds (radio static, fake talk radio)
Software Tech Stack
Pure Data - brains
sound for on-stage speakers, scheduling, cues
Node.js (Macchiato/Clojurescript) - messasge forwarder
host web page
OSC communication with Pure Data
WebSocket communication with individual clients (socket.io)
triggered events from Pd via OSC
Software Tech Stack cont.
HTML/CSS/JS(CLJS) - message implementer
client-facing view
React/Reagent/JS - reactive visuals/info display
Web Audio API/Tone.js - synths, samples
Websocket communication with server via socket.io
Hardware Tech Stack
laptop
audio interface (+ mics)
wireless router
Jobs
audio for on-stage speakers
live input recording (attack-triggered at max rate of 20s) / processing (rev./chopped, delays, FFT bin mag. thresholding)
samples + synthesis
user interface (controls, clocks, cues)
clocks for overall piece, current section, current section duration, next event timing
sec. 2 - cues for roll entrances 8th note before start and finish
Jobs cont.
scheduling and triggering
determine structure of piece (3 major sections)
communication with node.js server (OSC)
retrieve client list (base64)
direct server to trigger events (WebSockets)
Chance Operations
overall section durations
randomly chosen within ranges to add up to at least seven minutes
timings of events/subsections
approx. times with given ranges (smw-nextsubsec 50 7 -> 50 seconds in given section +/- 7)
perc. part cued as well! (sec. 2)
Chance Operations cont.
specific contents of events
random selection of given range of clients
random selection and playback location of sound files (sec. 2)
randomization of parameters of synthesized sounds (sec.1 drone)
random triggering (and selection of params within given ranges) of samples
randomization of delay lines (section 2)
all based around abstractions of [random] object (LCG)
Jobs
serve web page for clients
keep track of indiv. connections via WebSockets (socket.io)
trigger events on all devices (broadcast)
trigger events on a specific device (emit)
communicate with Pure Data (OSC)
provide list of connections (base64)
forward triggers from Pd to clients
Client (HTML/CSS/JS via CLJS)
Jobs
client-facing interface loaded on devices
CSS, JS for dynamic background, React (Reagent) for dynamic text
sound synthesis/playback (Web Audio API / Tone.js)
triggered by Node webserver (with visual feedback)
Chance Operations
all sequences generated client-side (audio.cljs)
chopped samples (sec. 1), filtered noise (sec. 1), telegraph beeps (sec. 3)
freq / dur / Q / sample location / amp. / etc.
start times of sequences (client.cljs)
flags from OSC for immed. start / delayed
other sound parameters
base freq. for modem synths (clicks/whine)
speed of modem clicks (subdivisions of 4s)
freq of sine osc. for telegraph
Technical Notes
why browser and not native?
easily (comparatively) cross-platform
no need to download additional software
server/client
cons
data payload size
limited to browser capabilities/non-native
Technical Considerations
accessing hosted web site?
static IP (easy, /etc/network/interfaces on *nix)
router and DNS server?
OpenWrt (DNS)
Dynamic DNS (host on computer with bind?)
qr-code direct to static ip
port number (port 80/custom port)
port 80 (Web): setcap 'cap_net_bind_service+3p' /usr/bin/nodejs
qr-code for static-ip addr. w/ page served on 8080
gen. w/ Racket Simple-Qr library (Chen Xiao)
Technical Considerations cont.
Apple/iOS - need user interaction to start audio context
latencies from WiFi (rhythmic/timing accuracy)
auto-screen lock without interaction
NoSleep.js - https://github.com/richtr/NoSleep.js
Further Exploration
not only for concert hall (installations)
streaming live sound - maybe Icecast/WebRTC/recording-loading sounds?
more sophisticated visualizations
P5.js, Canvas, Three.js, Quil, AR.js...
greater user interactivity
many options for input (text, accelerometers, sensors, GUI)
user control of sound, user supply of sound
interactions: user-server, user-user
Biblography
"[Met Performance] CID:49210." MetOpera Database: The Metropolitan Opera Archives, Metropolitan Opera Association, archives.metoperafamily.org/archives/scripts/cgiip.exe/WService=BibSpeed/fullcit.w?xCID=49210&limit=2500&xBranch=ALL&xsdate=&xedate=&theterm=1910-11&x=0&xhomepath=&xhome=. Archive of December 3, 1910 Matinee Program Information
Bibliography cont.
Kane, Joseph Nathan. Famous First Facts. The H.W. Wilson Company, 1964.
Fantel, Hans. "Sound; Out of De Forest And Onto The Air Came Music." The New York Times, 14 Jan. 1990, p. 002030, www.nytimes.com/1990/01/14/arts/sound-out-of-de-forest-and-onto-the-air-came-music.html.
Bibliography cont.
Sims, Michael. "Cavalleria Rusticana and I Pagliacci Crimes of Passion." Cavalleria Rusticana and I Pagliacci, www.concertoperaboston.org/cavalleria.html.
Guglielmo Marconi - Biographical. NobelPrize.org. Nobel Media AB 2018. Sun. 14 Oct 2018.
Resume presentation
Bates Digital Music Symposium 2018
socketMusic: wireless (2015)
for improvised percussion, electronic sound, and mobile devices
by Derek Kwan
http(s)://derekxkwan.com
https://derekxkwan.github.io/smwireless-slides/