First patient with new ‘mind-reading’ device uses brain signals to write
A person who developed paralysis and lost the ability to speak after a stroke can now communicate using a system that translates his brain’s electrical signals into individual letters, allowing him to read whole words in real time. and allows the formation of sentences.
To use the device, which receives signals from electrodes implanted in his brain, the man quietly attempted to say code words that stand for 26 letters of the alphabet, according to a new report published Tuesday (November 8) in the journal stand up. nature communication (opens in new tab), These code words come from the NATO phonetic alphabet, in which “alpha” stands for the letter A, “bravo” stands for B, and so on.
“The NATO phonetic alphabet was developed for communication over noisy channels,” Sean Metzger (opens in new tab), first author of the study and a doctoral candidate at the University of California, Berkeley and the University of California, San Francisco’s graduate program in bioengineering, told Live Science. “That’s the kind of situation we’re in, where we’re in this noisy environment of neural recordings.” The researchers initially tried to use individual letters instead of code words, but their system struggled to distinguish phonetically similar letters, such as b, d, p and g.
By silently speaking the NATO code words, the user generates Brain Activity that can then be decoded by algorithms that link together desired letters and insert spaces between words. To end a sentence, the user tries to squeeze his or her right hand; This produces specific brain activity that tells the device to stop decoding.
related: What happens in our mind when we ‘hear’ our thoughts?
In recent tests, the man could construct sentences from a vocabulary of more than 1,150 words at a speed of 29.4 characters per minute, or about seven words per minute. The decoder device occasionally made mistakes when translating their brain activity into letters, showing an average character error rate of 6.13%.
This marks an improvement from previous testing of the system, which was described in the 2021 report The New England Journal of Medicine (opens in new tab), In that test, the man constructed sentences by attempting to say whole words aloud from a prescribed vocabulary of 50 words. The device can decode about 18 words per minute with an average accuracy of 75% and a maximum accuracy of 93%.
“It was great, but limited,” in terms of vocabulary and how the user attempted to say the words aloud, Metzger said. The latest testing of the system shows that the system still works in silence and that by using a spelling approach, the user can greatly expand the available vocabulary. In the future, the two approaches can be easily combined: users can rely on the whole word decoder to quickly generate common words, and they can use a single-letter decoder to spell less-common words. Yes, Metzger explained.
The person featured in both studies is the first participant in Brain-Computer Interface Restoration of Arm and Voice (BRAVO) Trial (opens in new tab), which is being held at UC San Francisco. The trial is open to adults who have lost significant speech and motor control due to conditions such as stroke, amyotrophic lateral sclerosis (ALS) and muscular dystrophy.
At age 20, the participant suffered a severe stroke that cut off blood flow to a part of the brain called the pons. This structure acts as a bridge between the brain and spinal cord, and after his stroke, the participant lost all his ability to move his head, neck and limbs and all his ability to deliver intelligible speech. In general, man now communicates by using his limited head mobility to select letters on the screen using a physical pointer or a head-controlled cursor.
The man entered the Bravo trial at age 36, at which time he had surgery to place a mesh of 128 electrodes on the surface of his brain. Importantly, these electrodes sit on top of an area of the wrinkled cerebral cortex that controls the muscles of the vocal tract, instructing them to move and thus produce specific sounds. It also covers the area of the brain involved in moving the hands.
related: Can we think without using language?
For now, to connect to the decoder, the test participant must be physically plugged into the device via a port that sticks through skin of his skull. Ideally, in the future, the system will be completely wireless, Metzger said.
To calibrate the decoder, the researchers also asked the participant to attempt to silently say each of the NATO code words and to practice squeezing their right hand. Over time, they even made him pronounce arbitrary words and copy entire sentences, letter for letter. Eventually, after about 11 hours of training with the system, the man can spell out his basic sentences and answer specific questions.
One limitation of the system is that a 2.5-second time window is allotted for each character; In that time, the user silently says a code word, and the system records and decodes the resulting brain signals. Reducing that time window and making the decoding speed more flexible will be key to increasing the speed of the system, Metzger said.
Although the new study involved only one participant, it is “still a break-through study,” said jun wang (opens in new tab)An associate professor in the departments of speech, language and auditory sciences and neuroscience at the University of Texas at Austin. More research is needed to know whether the same approach will work for other patients, or if it will need to be adapted somewhat for each person, Wang told Live Science in an email.
Wang said that to be fit for daily use, such devices would need to be easy for patients and their caregivers to operate without assistance, and they would need to interface with other computer software. The technology would be particularly useful for patients in the closed state, who are almost completely paralyzed but retain all of their cognitive function and some ability to move their eyes, he said.
#patient #mindreading #device #brain #signals #write
