Seeing with sound

February 14, 2003 5:09 PM

See with your ears: the vOICe Sonification system. The vOICe Learning Edition translates arbitrary video images from a regular PC camera into sounds. [More inside.]
posted by Wet Spot (2 comments total)

This system takes an image and converts it to sound through a simple algorithm: the x-axis of the image represents time, the y-axis frequency, and the image intensity volume. It can be used as a seeing aid for the blind.

The vOICe Java applet is especially fun and has lots of examples. You can load an image or a mathematical formula and convert it to a "soundscape". Or you can load the visual equivalent of a Bach two-part invention (the image bears a striking resemblance to a piano roll), or the Shepard scale. It can even sonify animations. (If you have problems loading the applet, check your browser compatibility. It works with Netscape 4.7x and Mozilla 1.2.)

This applet has been a big hit in a freshman engineering course I teach - I use it to explain frequency domain concepts. I've had students switch majors after playing around with it.

The site is huge and not particularly well organized. The site map will help you get around.
posted by Wet Spot at 5:13 PM on February 14, 2003

Crikey--that's amazing. I guess that's sort of what it's like to be a dolphin or bat. (Er, a blind dolphin or bat.) Three cheers for technology!
posted by hippugeek at 8:00 PM on February 14, 2003