Jan 26, 2022
This podcast is about big ideas on how technology is
making life better for people with vision loss.
Every day, people who are blind or visually impaired
use their hearing to compensate for vision loss. But when we lose
our vision, can we access our visual cortex via other senses?
We call this ability for the brain to change its
activity “plasticity,” and brain plasticity is an area of active
research. In this episode, we’ll explore how, through
sensory substitution, audio feedback can, in some cases, stimulate
a user’s visual cortex, allowing a user to — without sight —
achieve something close to visual perception.
Erik Weihenmayer — world-class mountain climber,
kayaker, and founder of No Barriers who lost his vision as a
teenager due to retinoschisis — brings us to the summit of Everest
by describing what it sounds like. He explains how his hearing
helps him navigate his amazing outdoor adventures safely. We also
speak with Peter Meijer, the creator of The vOICe, an experimental
technology that converts visual information into sound, and has
been shown to activate users’ visual cortices, especially as users
train on the technology, and master how to interpret the audio
feedback. We hear an example of what users of The vOICe hear when
it translates a visual image of scissors into audio. Erik
Weihenmayer shares his experience with Brainport, a similar sensory
substitution technology featured in our episode “Training
the Brain: Sensory Substitution. While research is ongoing in the areas of sensory substitution
and brain plasticity, it’s encouraging that some users of The vOICe
report that the experience is like seeing. In the spirit of Erik
Weihenmayer, one user even uses it to surf.
The Big Takeaways:
- Erik Weihenmayer, despite having lost his vision as
a teenager, has become a world-class adventurer. He summited
Everest in 2001 and then summitted the highest peaks on each
continent. He has also kayaked 277 miles of whitewater rapids in
the Colorado River through the Grand Canyon. He explains how his
sense of hearing, in addition to his other senses, and
technologies, teams, and systems, helps him achieve his goal to
live a life with no barriers.
- Dutch Inventor Peter Meijer developed a technology
called The vOICe, which converts a two-dimensional image from a
camera into audio feedback. Dr. Roberts interviews Dr. Meijer about
this technology and gives listeners a chance to hear what The vOICe
sounds like. Users who train on this system interpret the sounds to
make sense of the original visual image. Research on The vOICe
shows that this happens in the brain’s visual cortex. While some
users say the experience is more auditory than visual, others
report the experience as akin to sight.
- The vOICe relies on the principles of sensory
substitution established by the founder of sensory substitution
Paul Bach-y-Rita. We discussed sensory substitution in our episode
the Brain: Sensory Substitution,” which featured
the Brainport device by WICAB. Erik has used Brainport, and in this
episode, he describes how the Brainport allowed him to catch a ball
rolling across a table, an exciting feat for someone who is blind.
He adds that sensory substitution takes serious practice to
- The vOICe is still in the experimental stage, and
more research has to be done on sensory substitution. However,
neuroscientists studying The vOICe have shown that it stimulates
the visual cortex, and some users report visual results.
- One user of The vOICe recently reported using the
technology to surf.
- “When there’s a lack of things that the sound
bounces off of, like on a summit, the sound vibrations just move
out through space infinitely and that’s a really beautiful
awe-inspiring sound.” — Erik Weihenmayer, No Barriers.
- “She rolled this white tennis ball across. It lit
up perfectly. [...] I’m like, ‘Holy cow, that is a tennis ball
rolling towards me.’ And I just naturally reached out and I grabbed
this tennis ball.” — Erik Weihenmayer, No Barriers (on using the
Brainport device by WICAB)
- “They applied transcranial magnetic stimulation to
... temporarily disrupt the processing and the visual cortex of a
user of The vOICe. ... So this showed that apparently, the visual
cortex was doing visual things again.” — Dr. Peter
Meijer, Seeing with Sound, The vOICe.
- “Some ... insist that the sensation of working with
the soundscape of The vOICe is truly visual. ... But .... most ...
users of The vOICe .... say, “It’s ... auditory but I can use it to
visually interpret things.” — Dr. Peter Meijer, Seeing with Sound,
- “Yeah, sure, if you want a really really safe life,
you can hang out on the couch and you can watch Netflix. But I
think most people want to be out there in the thick of things. They
want to be in the food fight.” — Erik Weihenmayer, No