Soldiers and stroke victims might one day have something in common: a device that allows them to talk without speaking. As this ScienCentral News video reports, NASA engineers are developing technology that picks up and translates throat signals into words before they're even spoken.

Silent Speech

Nancy Pinsker ticks off the names of five blue items, noting last the sky. She's better at this than she was five years ago when a stroke put a stranglehold on her vocal muscles, altering her speech. Since then, she's been chipping away at reclaiming the capacity for normal conversation.

"I feel if I don't come here and take lessons, I'll lose my voice," says the retired bookkeeper, now in her seventies. "Even now, it's not perfect."

For people like Pinsker who find it hard to engage in conversation, a host of new technology awaits. "There's just been an explosion," says Stephen Cavallo, a speech-language pathologist and associate professor at Lehman College, where Pinsker frequents the Speech and Hearing Center.

Now NASA researchers are taking a leap in the direction of deciphering speech. Neuroengineer Chuck Jorgensen told Discover Magazine that he's bypassing the physical body's normal requirements by delivering words via machine using subvocal speech. "When you're reading material…sometimes you find that your tongue or your lips are quietly moving but you're not making an audible sound," he explains. "And it's doing that because there's this electronic signal that's being sent to produce that speech but you're intercepting it so it doesn't really say it out loud. That's subvocal speech."

In a lab at NASA's Ames Research Center, electrodes similar to those used in a doctor's office cling below Jorgensen's chin and flank his Adam's apple, picking up electronic signals that the body sends to vocal chords. Jorgensen amplifies the signals and uses neural network software to decipher word patterns.

"We're starting with a small set of words—stop, go, left, right…or maybe the digits, zero through nine," he says. "We're not recognizing all possible speech. To answer that question, we have to find out whether we can recognize a smaller component than a word…like vowels and consonants."

Those sounds create waves that electrodes pick up and funnel into a neural net programmed with certain inputs. The net recognizes the pattern and the label—or word—that Jorgensen assigns to that pattern. Over time, word repetition and processing reduce errors, enabling researchers to introduce new patterns—or words—that the system hasn't seen, thus expanding the net's vocabulary, now at 50 words.

But Jorgensen can expect a huge hurdle: conversational speech is very different from uttering a word, says Cavallo. "I wonder how well the system will recognize subvocal speech during conversation and even at…the level of individual sounds," he says. "How well can it recognize a 'p' from a 'b', for example? How consistently can it do that?"

Given simple commands—words that Jorgensen considers small mathematical sets—the accuracy is "up to the high 90s to 100 percent. When we're talking about something like vowels and consonants, we're still in the 70 percent range. That's not too bad but we've got a ways to go yet," he says.

Still, the expectation that subvocal speech devices might retool communication is high. Applications could include communication in "covert operations with military troops, private speech between individuals, situations where you can't speak normally, underwater or in fire gear, or possibly where there's high noise or you have a respirator," Jorgensen says. He also envisions it blunting a modern pet peeve: loud, public cell phone use. One day, he says scientists will most likely develop a silent cell phone.

For speech-language pathologists like Cavallo and patients like Pinsker, the real excitement is in the technology's potential to impact quality of life in people suffering from speech ailments. Jorgensen says that if a person can pronounce an individual word consistently and it "translates into an electrical signal" that he can measure, he can "have that electrical signal be related to a word that anybody can understand."

"It's certainly exciting stuff," Cavallo says. "I guess a few years ago it might be the stuff of science fiction."

Jorgensen cautions that it's way too soon to tell how effective subvocal speech detection will translate medically. Instead, he offers to demonstrate what he knows it can already do: Before a wide screen, he maneuvers a simulated Mars Rover over Martian terrain. It dips, falls, climbs over craters and turns abruptly to the left and right, all at Jorgensen's prompting, all without him uttering a sound.

Consider it NASA's first giant leap towards deciphering subvocal speech.

Source