Technological and Computer-based Projects
PianoBots: a swarm of virally-musical robots with distributed intelligence
Proposed project
[PianoBots drawing]
- Each robot has two legs. They can jump.
- They can detect the lines made by the spaces between the keys, and also know about landing on white keys vs black keys.
- A robot can land with both legs on a single key to play that key, or can land with each foot straddling two keys to "walk" to a new location without pressing the keys down (its weight will be distributed across 2-4 keys this way, so there's a medium-sized range of body-weights each Bot could have)
- The height to which a robot jumps to will determine the velocity with which it lands, and therefore the dynamic of that note.
- Now, imagine a swarm of these robots... communicating their positions wirelessly.
- Each robot could either be calibrated to start on a certain note, or every robot could always start on the lowest (or highest) note of the piano, and then count its position from there. But it would really be coolest if each robot had a small microphone and "listened" to the sounds around it, deduced which of the sounds it was creating, and then knew which piano key it was on, based on that input.
- Now, if two robots want to play a scale, they can "team up" to jump past each other, playing the scale more quickly (or more loudly, or with greater accuracy) than could be achieved by a robot acting independently.
- They wouldn't even need to transmit much information between each other; based on their initial location each robot would know which notes it should take, for maximum efficiency, and which notes the other robot should take.
Both (all! but that's getting ahead of the story) robots would listen to the sounds being made by the other robot and be able to determine its position/speed/accuracy, and fine-tune its own actions accordingly.
(The exception to this would be if you started to get robots colliding in mid-air. They may have to talk to each other, a bit, to "reserve airspace". But then again, there might be an elegant solution in prototyping that I can't think of with all these Bots flying around only in my head, and not on an actual piano.
- Now, you're probably wondering: how will they know what to play? This would be the only part that would be centrally-controlled: a computer, a chip, something will have to radio-broadcast MIDI notes. But! No need to have each bot get an ID code that you prefix "its" notes with, no parsing program
to decide that Robot 17 gets note C4 in bar 31. All the notes get broadcast generally, and each bot determines which notes it can play, and goes for them. I suppose they'll have to shout out "got it!" to each other, like volleyball players, so that if two Bots can both reach a given note, they don't bash heads. But, especially once the piece has started,
much of that will be precluded simply because the Bots' note-grabbing logic will be able to take into account the position of the nearest couple of Bots, based on what notes they've been playing, calculate the distance it would have to travel, as opposed to the other Bot, and then easily work out which of them should "dive for it".
- The swarm, then, collaborates to play the piece. In the case of small swarms, some notes might get dropped. The swarm does its best, given its ability. (Look! Fuzzy ability in robots!)
- Obvious issues include, but are not limited to: latency. Imagining a Chopin piece that gradually works its way to the bottom of the piano, only to suddenly start another phrase from the highest register... well, some Bots are going to have to work their way up there, or stay up there, to play it in anything approaching musical time. With a big enough swarm, this gets easier, but it seems like it should be possible to have the Bots play and calculate, say, 100 notes behind the transmitter, so they have time to (a) do calculations, and (b) get into position.
