There are two parts to turning movement into music:
Part one is the motion tracking. Video cameras attached to a computer, analyze expressive shapes, movements and gestures and turn them into computer data.
Part two is called mapping. This means assigning the different body parts, gestures and movements to musical features, such as the notes of a piano.
But just playing notes is not the same as making music! This is where our composers come in. Using a kind of software called algorithmic composition, users are helped to play musically. For example, when you play a note, the software looks for notes that sound good together. The same is true of the player’s rhythm. If you play a little too early, the software will correct your mistake to keep you on the beat.
If done correctly, the player will have a feeling of hearing their body as music. This is our ultimate goal.