Waving Hello: Music Controlled by 3D Gestures

via Synthtopia

Y'know, there is just so much cool shit going on right now in gestural control that it seems criminal not to be involved in pointing it out to people. I've been so busy in my life that blogging began to feel like more of a luxury than a necessity. But I'm trying to carve out some time for it as there's so much to report.

With the above video, I feel like my quest has come full circle, or to the end of the road, or the beginning, or something, as pertains to my own interest in gestural music interfaces.

"3 Movements" ca. 1993

Part 3

For instance, here I am 20 years ago at art school, playing a gesture controller I built with Bill Tremblay, accompanying a talented trumpet player named Jeremey St Martin.

Part 1, in which I introduce the sound square with angelic choir samples.

Part 2, in which I trigger body-like sounds behind by a film projection by Catherine Hollander.

I made some arguably ok "music" with my gizmo. It was fresh off the slab, still wet from a layer of black spray-paint Bill had applied about an hour before the show. There were some truly sublime moments but none are recorded anywhere except in my memory.

The 1993 Sound Square was built on 10-year-old-at-the-time equipment, an Apple ][e (I think), with a MIDI card in it, 16 light sensors and emitters, some 2x3's, and a MIDI breakout board built by Bill's brother, Ben. The sounds came from an Ensoniq Mirage synthesizer. The code was written in Forth. The repeating notes are caused by keeping my hand in the beam - the duration of the cycle is equivalent to the processing time of the Forth loop that scanned the Apple's MIDI buss. So a quick "hit" then back out could be fast enough to trigger single notes or short stacatto bursts on each "string". It was also presented as an installation, cycling a set of sounds that viewers could trigger themselves, such as thunder and lightning, laughter, etc.

So now it's 2013 and FINALLY we have Kinect and now this extremely affordable Leap Motion thing and soon the technology will be built into phones and pads and laptops and we'll all be waving our hands around like dorks.
I'm pretty excited to be aware of and involved in this research, as I think it holds great promise. I think it's just the beginning, and we'll start to see some really amazing applications very soon.