Researchers at UC San Francisco and UC Berkeley have developed a brain-computer interface (BCI) that allows a woman with severe paralysis from a brainstem stroke to speak through a digital avatar.
This is the first time that speech or facial expressions have been synthesized from brain signals. The system can also decode these signals into text at a rate of almost 80 words per minute, a significant improvement over existing technology.
“Our goal is to restore the full, embodied way of communication that is the most natural way for us to talk to others,” said Edward Chang, chair of neurological surgery at UCSF.
Chang’s team has previously shown that a man who had also experienced a brainstem stroke years earlier could decode brain signals into text. The current study shows something more ambitious: decoding brain signals into speech with movements that animate a person’s face during a conversation.
Chang implanted a paper-thin rectangle of 253 electrodes on the surface of the woman’s brain in areas his team believed were critical for speech. The electrodes captured the brain signals that, in the absence of the stroke, would have gone to his muscles, his tongue, jaw and throat, as well as his face. A cable connected to a port attached to his head connected the electrodes to a bank of computers.
The woman spent weeks working with the team to train the system’s artificial intelligence algorithms to recognize her unique brain signals for speech. This involved repeating different phrases from a 1,024-word conversational vocabulary over and over until the computer recognized patterns of brain activity associated with the sounds.
Instead of teaching the AI to recognize whole words, the researchers created a system that decodes words from phonemes. They are the subunits of speech that make up the spoken word in the same way that letters make up words. For example, “Hello” contains four phonemes: “HH,” “AH,” “L,” and “OW.”
Using this approach, a computer only needed to learn 39 phonemes to decipher any English word. This increased the accuracy of the system and made it three times faster.
Source: Science Daily