Researchers with the BrainGate Collaboration have deciphered the brain activity associated with handwriting: working with a 65-year-old (at the time of the study) participant with paralysis who has sensors implanted in his brain, they used an algorithm to identify letters as he attempted to write them; then, the system displayed the text on a screen; by attempting handwriting, the participant typed 90 characters per minute more than double the previous record for typing with a brain-computer interface.

As part of the BrainGate clinical trial, researchers are using tiny electrode arrays to record signals from the motor cortex of the brain; those signals can then be used to control robotic prostheses, computers or other devices; the hope is that such a system may one help restore communication and movement in people with paralysis due to injury or illness. Image credit: BrainGate.org.

Brain-computer interfaces can restore communication to people who have lost the ability to move or speak.

So far, a major focus of brain-computer interface research has been on restoring gross motor skills, such as reaching and grasping or point-and-click typing with a computer cursor.

However, rapid sequences of highly dexterous behaviors, such as handwriting or touch typing, might enable faster rates of communication.

Scientists from the BrainGate Collaboration have been working for several years on such systems.

Previous studies have involved trial participants thinking about the motions involved in pointing to and clicking letters on a virtual keyboard. That system enabled one participant to type 40 characters per minute, which was the previous record speed.

For the latest study, the BrainGate researchers wanted to find out if asking a participant to think about motions involved in writing letters and words by hand would be faster.

An important mission of our BrainGate Consortium research is to restore rapid, intuitive communication for people with severe speech or motor impairments, said Professor Leigh Hochberg, a critical care neurologist in the School of Engineering and Carney Institute for Brain Science at Brown University, the Center for Neurotechnology and Neurorecovery at Massachusetts General Hospital, and the Department of Veterans Affairs Providence Healthcare System.

The new demonstration of fast, accurate neural decoding of handwriting marks an exciting new chapter in the development of clinically useful neurotechnologies.

We want to find new ways of letting people communicate faster, said Dr. Frank Willett, a neuroscientist at Stanford University and the Howard Hughes Medical Institute.

This new system uses both the rich neural activity recorded by intracortical electrodes and the power of language models that, when applied to the neurally decoded letters, can create rapid and accurate text.

As part of the clinical trial, the scientists placed two tiny electrodes about the size of a baby aspirin in a part of the trial participants brain associated with the movement of his right arm and hand.

Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter.

With this system, the man could copy sentences and answer questions at a rate similar to that of someone the same age typing on a smartphone.

The system is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another, Dr. Willett said.

The innovation could, with further development, let people with paralysis rapidly type without using their hands, said Dr. Krishna Shenoy, a researcher at Stanford University.

This technology and others like it have the potential to help people with all sorts of disabilities, added Dr. Jose Carmena, a neural engineer at the University of California, Berkeley.

Though the findings are preliminary, its a big advancement in the field.

Brain-computer interfaces convert thought into action. This paper is a perfect example: the interface decodes the thought of writing and produces the action.

The people who enroll in the BrainGate trial are amazing, Professor Hochberg said.

Its their pioneering spirit that not only allows us to gain new insights into human brain function, but that leads to the creation of systems that will help other people with paralysis.

The teams work was published in the journal Nature.

_____

F.R. Willett et al. 2021. High-performance brain-to-text communication via handwriting. Nature 593, 249-254; doi: 10.1038/s41586-021-03506-2

See the article here:
Brain-Computer Interface Translates Brain Signals Associated with Handwriting into Text | Computer Science, Neuroscience - Sci-News.com