Hearing Through Your Fingers: Device that Converts Speech to Touch Enables the Hearing-Impaired to Hear Through Their Fingertips
“Despite recent advancements in hearing aid and cochlear implants, the most widespread surgical implant to restore audition, hearing-impaired users still encounter significant practical and social challenges with or without aids,” explained lead investigator Amir Amedi, PhD, Department of Medical Neurobiology, Institute for Medical Research Israel-Canada, Faculty of Medicine, Hebrew University of Jerusalem, Hadassah Ein-Kerem, and The Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, Israel. “In particular, they all struggle with understanding speech in challenging, noisy acoustic environments, especially in presence of a competing speaker.”
The number of sensory deprived patients and auditory deprived patients is expected to rise so it is crucial to develop efficient techniques for auditory recovery designed to convey the missing information to patients. “We live in a world that is becoming steadily more multisensory and we really need to understand the mechanisms underlying multisensory perception and integration. Providing relevant information using the sense of touch can significantly help hearing,” commented Dr. Amedi.
In this current proof-of-concept study investigators hypothesized that they would be able to improve speech understanding under challenging conditions by exploiting the ability of the brain to integrate information coming simultaneously from different senses. They designed a minimalistic auditory-to-tactile sensory substitution device (SSD) that transforms low-frequency speech signals into tactile vibrations delivered on two fingertips. They asked a group of non-native English speakers to repeat a series of sentences, which was degraded by embedding them in speech-like noise.
As expected, when participants could rely only on audition, their understanding of such sentences was poor. Crucially, however, their sentence understanding significantly improved when they paired the degraded speech signal with complementary vibration delivered on the participants fingertips. The vibration conveyed a specific set of frequencies known as fundamental frequencies that characterize speech signals.
Caption: A) Vibrating interface of the Vibrating Auditory Stimulator; B) Matlab GUI for stimuli presentation and control; C) Speech Reception Threshold (SRT) values obtained for auditory and auditory-tactile speech in noise stimulation at the group level; and D) in individual subjects [subject 6 showed an improvement from 0.3 to –3.0 SRT(dB)]
Credit: IOS Press
The reported improvement at the group level was 6 dB – a major difference considering that an increase of 10 dB represents a doubling of the perceived loudness. These results are especially relevant when compared to earlier SSD studies showing effects in behavior only after demanding cognitive training.
“Our results carry important implications for further research, as well as possible clinical and practical solutions,” noted co-author Tomasz Wolak, PhD Eng, Head of the Bioimaging Research Center, Institute of Physiology and Pathology of Hearing, World Hearing Center, Warsaw, Poland. “The ability to ‘hear through one’s fingers can significantly help hearing. Our approach suggests that multisensory stimulations providing the same type of information (in this case spoken language conveyed through touch in addition to hearing) should be processed in the same brain region (in this case spoken language centers), ultimately then predicting that multisensory stimulations (both sounds and touch) should enhance perception.”
According to lead author Katarzyna Cieśla, PhD, World Hearing Center, Warsaw, and Hebrew University of Jerusalem, “The most compelling aspect of our study is the fact that learning to use such speech-to-touch sensory substitution device did not require any training. We believe it can also serve as an aid for the elderly population, which finds it challenging to follow extensive training regimes. This might be the first study showing such immediate relevant enhancement of a sensory substitution device and suggests the brain is much more multisensory than the common wisdom.”
“This study is a major step forward to introduce multisensory plasticity of the brain as an innovative paradigm to maximize the potential of patients to compensate for their sensory loss,” commented Bernhard Sabel, PhD, Editor-in-Chief of Restorative Neurology and Neuroscience.
Next the team plans to further improve the device and training regimes in order to reach the goal of 10 dB enhancement and to test for human brain mechanisms using an MRI-compatible version of the device in various populations (both hearing and hearing-impaired people).
Citation: “Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution,” by Katarzyna Cieśla, Tomasz Wolak, Artur Lorens, Benedetta Heimler, Henryk Skarżyński, and Amir Amedi. Published in Restorative Neurology and Neuroscience, Volume 37, Issue 2, DOI: 10.3233/RNN-190898, by IOS Press. This article is openly available at content.iospress.com/articles/restorative-neurology-and-neuroscience/rnn190898.
Get our Free Ebook, "Suppressed Health Secrets" with Natural Cures THEY don't want you to know!
APeX - Far superior to colloidal silver! Destroys Viruses, Bacteria, Pathogens with Oxygen plus Silver!Supreme Fulvic - Nature's most important supplement! Vivid Dreams again!
Ultimate Curcumin - Natural pain relief, reduce inflammation and so much more.
MitoCopper - Bioavailable Copper destroys pathogens and gives you more energy. (See Blood Video)
Oxy Powder - Natural Colon Cleanser! Cleans out toxic buildup with oxygen!
Organic Hemp Extract (CBD) - Full Spectrum high CBD (3300mg) hemp extract eases stiff joints, relieves stress and more!
Nascent Iodine - Promotes detoxification, mental focus and thyroid health.
Smart Meter Cover - Reduces Smart Meter radiation by 96%! (See Video)
FINAL WARNING! Diseases are EXPLODING! Watch this Video about APeX and You'll THROW AWAY Your Colloidal Silver! APeX destroys Viruses, Bacteria and other Pathogens with the power of Oxygen PLUS Silver! Nobody else has a product like THIS! See why the inventor hasn't been sick in 16 years and why you'll never hear about it on the FAKE NEWS! Get some now and tell your friends about it too so we can reach more people!