Full Article
Peter Meijer, who works at Philips Research in The Netherlands, has developed software that grabs images from any run-of-the-mill WebCam
and translates the video frames into complex sounds, or "soundscapes". The idea is that blind people may learn to interpret these soundscapes
and thus learn to see with their ears. The approach is coined The vOICe,
with the capitals "O I C", when spoken out loud, hinting at the intended purpose: "Oh, I see!". The principle underlying the soundscapes
is simple: images are scanned from left to right while the vertical position of every pixel is indicated by pitch and its brightness by
corresponding loudness.
For instance, click on the image below to compare it to its corresponding one-second (22K) soundscape. Can you figure it out and hear the
curved line and the ten little white squares? Just play the sound a number of times until you "get the picture".
Obviously, with real-life images these soundscapes can become exceedingly complex, and learning to interpret arbitrary visual scenes through sound
will require extensive training, perhaps comparable to learning a foreign language. It is not yet known how well blind people can learn to do this.
Want a more realistic example to get a flavour of what the visual environment can sound like? Then click on the photograph below to hear its one-second
(44K) soundscape, and you will hear the building with the big white area in the middle sounding as a kind of noise burst and the rows of windows
as a fast rhythm on your right side. The use of stereo headphones is strongly recommended.
Now many of you may already have a WebCam, so why not just give this seeing-with-sound software a try yourself and listen to your own live
WebCam view. The software, called The vOICe Learning Edition, can be downloaded for free (for non-commercial purposes) from Peter Meijer's
web page:
http://www.seeingwithsound.com/winvoice.htm (or the mirror page)
The file size of the executable is less than 500K and it runs under Microsoft Windows-95/98/ME/NT/2000.
Some blind people have started using this software with a head-mounted WebCam, stereo headphones, and a notebook PC stuffed inside a backpack.
This is wearable computing for real! Maybe it would be fun to demonstrate your WebCam with this software to a blind friend or relative? Pressing
function key F10 with this software makes it double as a talking color identifier. Enjoy!
[Image and sound samples on this page: courtesy of Peter Meijer]
Review by:
