"Neural signals from volunteers who imagine speaking are directly translated into audible output by our speech neuroprosthetic - in real time with no perceptible latency!" The exciting research result has now been published in the prestigious scientific journal Nature Communications Biology.
The innovative speech neuroprosthetic is based on a closed-loop system that combines technologies from modern speech synthesis with brain-computer interfaces. This system was developed by Miguel Angrick at the CSL. As input, it receives the neural signals of users who imagine speaking. Using machine learning, it translates them into speech almost immediately and outputs audible feedback to its users. "This closes the loop for them from imagining speaking to hearing their speech," says Angrick.
The work published in Nature Communications Biology is based on a study with a volunteer epilepsy patient who was implanted with depth electrodes for medical examinations and was in hospital for clinical monitoring. In the first step, the patient read texts aloud, from which the closed-loop system learned the correspondence between speech and neural activity by means of machine learning. "In the second step, this learning process was repeated with whispered and imagined speech", explains Miguel Angrick. "In the process, the closed-loop system produced synthesised speech. Although the system had learned the correspondences exclusively on audible speech, audible output is also produced with whispered and imagined speech." This suggests that the underlying speech processes in the brain for audibly produced speech share to some extent a common neural substrate to those for whispered and imagined speech.
"Speech neuroprosthetics focuses on providing a natural communication channel for people who are unable to speak due to physical or neurological impairments," says Professor Tanja Schultz, explaining the background for the intensive research activities in this field, in which the Cognitive Systems Lab at the University of Bremen plays a world-renowned role. "Real-time synthesis of acoustic speech directly from measured neural activity could enable natural conversations and significantly improve the quality of life of people whose communication capabilities are severely limited."
MEDICA-tradefair.com; Source: University of Bremen
