Technological and Computer-based Projects

eResonators
proposed project/product

Digital simulations are impressive, we can all agree on that. Whether an imaginary glass string being bowed in air morphing smoothly into a steel whistle being softly struck in treacle, the range of possibilities is endless. But at a certain point, they ultimately lack some dirt, some grit, some imperfection that's automatically part of anything that exists in physical space: even the best violin in digital space will still lack this feature that you can get simply from stretching a rubber band over a shoebox. Animators refer to this as the "hair paradox": in digital animation it's easy to blow up a city, but if the script calls for one character to touch another's hair the animation team will be pulling their hair out.

eResonators is a plan to "sample" physicality from real-world objects, being resonated and re-sampled within a small USB device. Through algorithms to be developed, the "physical" sample will be convolved with the digital simulation to bring individualized, characteristic realism to digital instruments. With eResonators, your instruments are again truly unique— no one else can quite recreate your cello (or your steel whistle in treacle).

The basic idea for this model stems from high-end audio mastering software/hardware combinations such as Universal Audio's UAD-2 Powered Plug-ins. These are software plug-ins that require specific hardware connected via PCI or PCMCIA port to the host computer. Critically, though, these differ in that the proprietary hardware only hosts further DSP on specialized chips: eResonators proposes to do something closer to hosting an actual optical compression circuit outside of the computer, but with a lot more flexibility.

Some inventive instrument builders have made computer-controlled instruments, MIDI robots whose sound can be recorded. Chris Beckstrom, a Chicago-area jazz artist and film composer, has made a miniature found-object drumset he uses to make beats, recording loops he plays via MIDI from Ableton Live. To my mind, these have potential as bespoke individual pieces, but for the most part are not suited to mass-production. What an artist will want is something flexible, something artistry and inventiveness can be brought to with fruitful results. Roberto Aimi, in discussing his Hybrid Percussion, emphasizes the importance of a predictable, continuous response from any aspect of a novel instrument; eResonators will provide a predictable, continuous though nonlinear relationship between parameters and heard output.

eResonators will each contain a small piece of a natural substance: Wood is the first that comes to mind, and stems from Amit Zoran's Chameleon Guitar, which was part of the original inspiration for this project. eResonators can, in fact, be seen as a broadening of his idea beyond guitars, and beyond performer-time into studio-time and beyond. Metal, glass, nylon, and stone are other strong possibilities, with varieties of each having additional potential. The "input" to the device will be fed into the physical medium, causing it to resonate according to the characteristics of the incoming sound. Its response will be sampled and sent back as the "output" of the plug-in.

One key aspect of marketing this technology, and keeping power consumption low, will be overall device size. For this reason and others to be delineated below, I propose to use ultrasound for all vibration and detection in the physically resonating medium. The frequency of the sound being sent from the computer will be multiplied by some constant— perhaps manually configured by the user, perhaps determined by the software in an initial "setup" or "training" phase, or perhaps simply fixed— and then the vibrations picked up by the reference mics will be divided by that constant to give back sound at the proper pitch level, but containing spectrally interesting characteristics in bands to which the human ear is particularly attentive.

Challenges foreseen

Because of the small size of these units, separating the input and output streams at close range will be difficult. Also, if off-the-shelf mics proved insufficient and specialized mics are needed, they could add expense to the manufacturing process. Last, researching the best range of ultrasonic frequencies to use could be a difficult process, and it is as yet impossible to estimate the amount of time it would take to produce something for public consumption. However, I see this is an exciting possibility to add something new to the fundamental way musicians and producers use computer technology to create and influence sound.