{"id":12549,"date":"2024-09-03T10:00:00","date_gmt":"2024-09-03T10:00:00","guid":{"rendered":"https:\/\/modernsciences.org\/staging\/4414\/?p=12549"},"modified":"2024-08-23T06:16:03","modified_gmt":"2024-08-23T06:16:03","slug":"from-thoughts-to-words-how-ai-deciphers-neural-signals-to-help-a-man-with-als-speak","status":"publish","type":"post","link":"https:\/\/modernsciences.org\/staging\/4414\/from-thoughts-to-words-how-ai-deciphers-neural-signals-to-help-a-man-with-als-speak\/","title":{"rendered":"From thoughts to words: How AI deciphers neural signals to help a man with ALS speak"},"content":{"rendered":"\n
\n
\n \n
\n Casey Harrell, who has ALS, works with a brain-computer interface to turn his thoughts into words.\n Nicholas Card<\/span><\/span>\n <\/figcaption>\n <\/figure>\n\n Nicholas Card<\/a>, University of California, Davis<\/a><\/em><\/span>\n\n

Brain-computer interfaces are a groundbreaking technology that can help paralyzed people regain functions they\u2019ve lost, like moving a hand. These devices record signals from the brain and decipher the user\u2019s intended action, bypassing damaged or degraded nerves that would normally transmit those brain signals to control muscles. <\/p>\n\n

Since 2006<\/a>, demonstrations of brain-computer interfaces in humans have primarily focused on restoring arm and hand movements by enabling people to control computer cursors<\/a> or robotic arms<\/a>. Recently, researchers have begun developing speech brain-computer interfaces<\/a> to restore communication for people who cannot speak. <\/p>\n\n

As the user attempts to talk, these brain-computer interfaces record the person\u2019s unique brain signals associated with attempted muscle movements for speaking and then translate them into words. These words can then be displayed as text on a screen or spoken aloud using text-to-speech software.<\/p>\n\n

I\u2019m a reseacher<\/a> in the Neuroprosthetics Lab<\/a> at the University of California, Davis, which is part of the BrainGate2<\/a> clinical trial. My colleagues and I recently demonstrated a speech brain-computer interface that deciphers the attempted speech of a man with ALS,<\/a> or amyotrophic lateral sclerosis, also known as Lou Gehrig\u2019s disease. The interface converts neural signals into text with over 97% accuracy. Key to our system is a set of artificial intelligence language models \u2013 artificial neural networks that help interpret natural ones.<\/p>\n\n

Recording brain signals<\/h2>\n\n

The first step in our speech brain-computer interface is recording brain signals. There are several sources of brain signals, some of which require surgery to record. Surgically implanted recording devices can capture high-quality brain signals because they are placed closer to neurons, resulting in stronger signals with less interference. These neural recording devices include grids of electrodes placed on the brain\u2019s surface or electrodes implanted directly into brain tissue.<\/p>\n\n

In our study, we used electrode arrays surgically placed in the speech motor cortex, the part of the brain that controls muscles related to speech, of the participant, Casey Harrell. We recorded neural activity from 256 electrodes as Harrell attempted to speak.<\/p>\n\n

\n \"A<\/a>\n
\n An array of 64 electrodes that embed into brain tissue records neural signals.<\/span>\n UC Davis Health<\/span><\/span>\n <\/figcaption>\n <\/figure>\n\n

Decoding brain signals<\/h2>\n\n

The next challenge is relating the complex brain signals to the words the user is trying to say.<\/p>\n\n

One approach is to map neural activity patterns directly to spoken words. This method requires recording brain signals corresponding to each word multiple times to identify the average relationship between neural activity and specific words. While this strategy works well for small vocabularies, as demonstrated in a 2021 study with a 50-word vocabulary<\/a>, it becomes impractical for larger ones. Imagine asking the brain-computer interface user to try to say every word in the dictionary multiple times \u2013 it could take months, and it still wouldn\u2019t work for new words.<\/p>\n\n

Instead, we use an alternative strategy: mapping brain signals to phonemes, the basic units of sound that make up words. In English, there are 39 phonemes, including ch, er, oo, pl and sh, that can be combined to form any word. We can measure the neural activity associated with every phoneme multiple times just by asking the participant to read a few sentences aloud. By accurately mapping neural activity to phonemes, we can assemble them into any English word, even ones the system wasn\u2019t explicitly trained with.<\/p>\n\n

To map brain signals to phonemes, we use advanced machine learning models. These models are particularly well-suited for this task due to their ability to find patterns in large amounts of complex data that would be impossible for humans to discern. Think of these models as super-smart listeners that can pick out important information from noisy brain signals, much like you might focus on a conversation in a crowded room. Using these models, we were able to decipher phoneme sequences during attempted speech with over 90% accuracy.<\/p>\n\n

\n