Press Release

06.11.07Electronic Music Instrument Reactable Turns Images to Sounds

Digital image processing opens up unlimited possibilities to artists, as demonstrated by the experimental musical instrument Reactable, developed by researchers from the University of Barcelona using cameras from Allied Vision Technologies. The new instrument on display at VISION 2007 is being successfully used by the international star Björk on her Volta tour.

Pop singer Björk made a name for herself years ago with her ethereal voice and avant-garde electronic sound. For her current Volta tour, the trailblazing Icelandian has discovered the creative potential of a new electronic musical instrument: the Reactable. This new style of synthesizer was thought up by a research group at the Pompeu Fabra University in Barcelona, Spain and relies on digital image processing.
 
Musical Amoebae
The Reactable looks like an illuminated, round table with a diameter of about 1 m (3 ft.) and a glass top. At first glance, the musical instrument of the 21st century does not make such a futuristic and complicated impression. And it is exactly this that is the secret of the Reactable: The interaction between musician and instrument takes place simply and intuitively via the smooth surface of the table on which the artist places and moves different objects in relationship to each other. No buttons, no switches, no keys.

To operate the Reactable, various Plexiglas objects are placed on the table, related to each other and moved. These objects fulfill different functions based upon their geometric shape. For example, square shaped elements generate basic tones, while round objects act as sound filters, which modulate these basic tones. The symbol on the selected elements determines the type of the basic tone and/or the filter; the spatial relationship of the objects to each other determines the extent to which one element affects another. A special collection of symbols was conceived for the Reactable which meet the system requirements for easy and fast recognition. Due to the fact that they look like one celled organisms, the symbols were given the nickname "amoebae" by the researchers.

The Reactable projects markings onto the surface of the table to make the instrument easier to operate. These not only confirm to the musician that the object has been recognized by the system, but also provide additional information regarding the status of the generated tone and its interaction with neighboring objects. This allows the artist to see the connections and a dynamic graphic presentation of the generated sound waves on the table. The kicker: The musician can change individual sound parameters by touching the projected information with his finger.

Digital Image Processing Turns Images into Sounds
The Reactable has ambitious image processing to thank for the fact that it is so easy and fun to use: An Allied Vision Technologies digital camera monitors everything that takes place on the table from underneath the glass plate. The specially developed image processing system ReacTIVision analyzes the images and derives corresponding sound information from the position of the objects. This is then transferred to the speakers as an audio signal and graphically projected onto the tabletop.

Alternative technologies would have been available for this project for determining the position of the objects on the surface of the table – such as ultrasound or RFID. Digital image processing prevailed, because this technology offers significant advantages, says Sergi Jordà, director of the project at the Pompeu Fabra University of Barcelona: “With a camera and a constant bandwidth, we can monitor the entire surface, regardless of how many objects are on the table. In addition, image processing determines not only the position of the objects but also their orientation (rotation) and can recognize human fingers on the tabletop. With this, the optical solution offers many more fine tuning possibilities and nuances than alternative technologies and places almost no limits on musical creativity."

What was critical for implementation, however, were the image transfer and processing speeds, as well as the sensitivity of the camera for the environmental lighting conditions. The instrument must react very quickly to the commands of the musician, especially if it is meant to play in synchronicity with a band. “With the digital FireWire cameras by Allied Vision Technologies, we have a high data rate with more than 60 frames per second available to us, which makes the short reaction time of the instrument possible”, says Jordà. The stability and reliability of the system is also extremely important: A breakdown during a concert by an international star like Björk is out of the question. “The AVT cameras are also impressive in terms of lifetime and stability", say the researchers. In order to lessen the sensitivity to environmental light, the AVT cameras work in the near infrared spectrum (NIR). A special LED light is used in conjunction with a filter on the camera for this.

Hard Test on World Tour
The University of Barcelona team was locally advised and supported for the integration of the AVT cameras by Allied Vision Technologies' Spanish distribution partner Infaimon, which is also located in the Catalan capital.
The Reactable is not yet being mass produced, but the actual prototypes have proven that the system can also function reliably in a live concert. As the first famous customer, Björk implements a Reactable on the world tour of her new album “Volta”. In this way not only does a wider audience have the chance to get to know the Reactable; the technical reliability of the system is also going to be subjected to a hard test. The Volta Tour 2007 with the Reactable will have concerts in Amsterdam, Barcelona, Baton Rouge, Brussels, Bucharest, Lisbon, Madrid, Milan, New York and Rome, among others. The goal of the inventors is to use the public attention to market the innovative instrument and begin to mass produce it.

More information about the Reactable at http://mtg.upf.edu/reactable/