Sound Rather than Sight Can Activate ‘Seeing’ for the Blind

Photo: Dr. Amedi wearing one of the SSD

The images are converted into
“soundscapes,” using a
predictable algorithm, allowing
the user to listen to and then
interpret the visual information
coming from the camera;
University of Jerusalem

SSDs are non-invasive sensory aids that provide visual information to the blind via their existing senses. For example, using a visual-to-auditory SSD in a clinical or everyday setting, users wear a miniature video camera connected to a small computer (or smart phone) and stereo headphones.

The images are converted into “soundscapes,” using a predictable algorithm, allowing the user to listen to and then interpret the visual information coming from the camera.

Remarkably, proficient users who have had a dedicated (but relatively brief) training as part of a research protocol in the laboratory of Doctor Amir Amedi, of the Institute for Medical Research Israel-Canada at the Hebrew University, are able to use SSDs to identify complex everyday objects, locate people and their postures, and read letters and words.

In addition to SSDs’ clinical opportunities, using functional magnetic resonance imaging opens a unique window for studying the organisation of the visual cortex without visual experience by studying the brain of congenitally blind individuals.

The results of the study in Amedi’s lab are surprising. Not only can the sounds, which represent vision, activate the visual cortex of people who have never seen before, but they do so in a way organised according to the large-scale organisation and segregation of the two visual processing streams.

For the past three decades, it has been known that visual processing is carried out in two parallel pathways. The ventral occipito-temporal “what” pathway, or the “ventral stream,” has been linked with visual processing of form, object identity and colour. Its counterpart is considered to be the dorsal occipito-parietal “where/how” pathway, or the “dorsal stream,” which analyses visuo-spatial information about object location and participates in visuo-motor planning.

Although this double dissociation between the processing of the two streams has been thoroughly validated, what remained unclear was the role of visual experience in shaping this functional architecture of the brain. Does this fundamental large-scale organisational principle depend on visual experience?

Using sensory substitution, the Hebrew University scientists discovered that the visual cortex of the blind shows a similar dorsal/ventral visual pathway division-of-labour when perceiving sounds that convey the relevant visual information. For example, when the blind are requested to identify either the location or the shape of an SSD “image,” they activate an area in the dorsal or in the ventral streams, respectively.

This shows that the most important large-scale organisation of the visual system into the two streams can develop at least to some extent even without any visual experience, suggesting instead that this division-of-labour is not at all visual in its nature.; Source: Hebrew University of Jerusalem