Jef's web files - Human interface

Non-acoustic sensors detect speech without sound

Sun, 10 Apr 2005 19:57:04 -0400

Just think how eerie it would be, yet also how peaceful - people all around having conversations on their mobile phones, but without uttering a sound.

Thanks to some military research, this social nirvana just might come true. DARPA, the US Department of Defense's research agency, is working on a project known as Advanced Speech Encoding, aimed at replacing microphones with non-acoustic sensors that detect speech via the speaker's nerve and muscle activity, rather than sound itself.

One system, being developed for DARPA by Rick Brown of Worcester Polytechnic Institute in Massachusetts, relies on a sensor worn around the neck called a tuned electromagnetic resonator collar (TERC). Using sensing techniques developed for magnetic resonance imaging, the collar detects changes in capacitance caused by movement of the vocal cords, and is designed to allow speech to be heard above loud background noise.

Scientists Discover What You Are Thinking

Sun, 20 Mar 2005 18:13:51 -0500

By decoding signals coming from neurons, scientists at the California Institute of Technology have confirmed that an area of the brain known as the ventrolateral prefrontal cortex (vPF) is involved in the planning stages of movement, that instantaneous flicker of time when we contemplate moving a hand or other limb. The work has implications for the development of a neural prosthesis, a brain-machine interface that will give paralyzed people the ability to move and communicate simply by thinking.

By piggybacking on therapeutic work being conducted on epileptic patients, Daniel Rizzuto, a postdoctoral scholar in the lab of Richard Andersen, the Boswell Professor of Neuroscience, was able to predict where a target the patient was looking at was located, and also where the patient was going to move his hand. The work currently appears in the online version of Nature Neuroscience.

Brainwave interface goes 2D

Wed, 09 Feb 2005 09:46:35 -0500

It would be useful to be able to move a cursor around a computer screen just by thinking about where you wanted it to go.

Researchers have been working on the task for more than a decade; some recent progress involved monkeys moving a cursor to specific spots on a computer screen. The price of being able to do this was fairly high, however -- electrodes implanted in the brain.

Researchers from the New York State Department Of Health and the Massachusetts Institute of Technology have shown that it is possible to use brainwaves picked up by electrodes attached to the outside of the scalp to move a cursor around a computer screen. Previous efforts at monitoring brain waves through the scalp only allowed movement along one axis, like up and down.

The Noninvasive Brain-Computer Interface allows people to learn to move a cursor around a two-dimensional computer screen by controlling the electrical noise that the brain makes as it functions, said Jonathan Wolpaw, chief of the Laboratory of Nervous System Disorders at the Wadsworth Center of the New York State Department of Health. The keys to the system are recent improvements in signal processing and an algorithm that adjusts to the way an individual controls brainwave oscillations.

NASA Extension of the Human Senses project

Sun, 12 Dec 2004 17:28:01 -0500

The goal of the Extension of the Human Senses project is to advance man machine interfaces by directly connecting a person to a computer via the human electrical nervous system. This involves measuring Electromyogram (EMG) and Electroencephalogram (EEG) signals and applying intelligent pattern recongition software to interpret these signals as computer control commands.
To date we have used EMG signals to eliminate the need for mechanical joysticks and keyboards. As an example of this we have flown a Class IV simulation of a transport aircraft to landing with our EMG based "joystick". We have also demonstrated virtual typing on a keypad using EMG. Our current work is focusing on using brain waves (EEG) to control computer software and the necessary algorithms to support this work.

Secret Speech Aid

Tue, 23 Nov 2004 10:11:26 -0500

Soldiers and stroke victims might one day have something in common: a device that allows them to talk without speaking. As this ScienCentral News video reports, NASA engineers are developing technology that picks up and translates throat signals into words before they're even spoken.

Silent Speech

Nancy Pinsker ticks off the names of five blue items, noting last the sky. She's better at this than she was five years ago when a stroke put a stranglehold on her vocal muscles, altering her speech. Since then, she's been chipping away at reclaiming the capacity for normal conversation.

"I feel if I don't come here and take lessons, I'll lose my voice," says the retired bookkeeper, now in her seventies. "Even now, it's not perfect."

For people like Pinsker who find it hard to engage in conversation, a host of new technology awaits. "There's just been an explosion," says Stephen Cavallo, a speech-language pathologist and associate professor at Lehman College, where Pinsker frequents the Speech and Hearing Center.

Now NASA researchers are taking a leap in the direction of deciphering speech. Neuroengineer Chuck Jorgensen told Discover Magazine that he's bypassing the physical body's normal requirements by delivering words via machine using subvocal speech. "When you're reading material…sometimes you find that your tongue or your lips are quietly moving but you're not making an audible sound," he explains. "And it's doing that because there's this electronic signal that's being sent to produce that speech but you're intercepting it so it doesn't really say it out loud. That's subvocal speech."

In a lab at NASA's Ames Research Center, electrodes similar to those used in a doctor's office cling below Jorgensen's chin and flank his Adam's apple, picking up electronic signals that the body sends to vocal chords. Jorgensen amplifies the signals and uses neural network software to decipher word patterns.

Patients play computer games using only the signals from their brains

Thu, 14 Oct 2004 09:09:53 -0400

For the first time in humans, a team headed by researchers at Washington University in St. Louis has placed an electronic grid atop patients' brains to gather motor signals that enable patients to play a computer game using only the signals from their brains.

The use of a grid atop the brain to record brain surface signals is a brain-machine interface technique that uses electrocorticographic (ECoG) activity-data taken invasively right from the brain surface. It is an alternative to the status quo, used frequently studying humans, called electroencephalographic activity (EEG) - data taken non-invasively by electrodes outside the brain on the skull.

Nose-steered mouse could save aching arms

Sun, 19 Sep 2004 16:01:46 -0400

Tired of using a mouse to control your PC? Perhaps there is another option for when your arm starts to ache: your nose. A novel PC control system lets users nudge a cursor around the screen with gentle movements of their nose. Blinking the left or right eye twice takes the place of left or right mouse clicks.

The inventor, Dmitry Gorodnichy of the Institute of Information Technology in Ottawa, Canada, calls his nose-steered mouse a "nouse". In addition to giving people a change from the keyboard and mouse, he hopes it will make using a PC easier for people who have a disability.

And it could also provide more intuitive ways for people to explore computer-generated environments or play three-dimensional video games, he says in the journal Image and Vision Computing (vol 22, p 931).

High-Tech Hearing Bypasses Ears

Sat, 18 Sep 2004 12:20:47 -0400

A wristwatch phone that lets you listen by sticking a finger in your ear, an MP3 player that vibrates the bones in your skull to play music that only you can hear -- these are some of the products being developed using a technology called bone conduction that sends sound waves through the bones around the ear.

Bone-conduction technology has long been used in hearing aids and other products for the hearing impaired, as well as in military headsets. Recently, several commercial companies have embraced it for products aimed at the general public.

The Right Ear Is From Mars

Tue, 14 Sep 2004 09:50:58 -0400

Belting out a few notes on key might take years of practice, and perfect pitch the right genetics, but when it comes to something as simple as telling noise from symphony, speech from music, all ears are created equal - or so it was once thought.

But in a new study, scientists have found that the left and right ears process sound differently. From birth, the right ear responds more to speech, while the left ear is more attuned to music, according to the study, published in Science on Sept. 10.

"Virtual clay" brings act of sculpting to the virtual world

Mon, 02 Aug 2004 11:20:48 -0400

Researchers from UB's Virtual Reality Lab have developed a new tool for transmitting physical touch to the virtual world.

Their virtual clay sculpting system enables users to replicate in real time on a personal computer the physical act of sculpting a block of clay or other malleable material. The resulting 3-D electronic shape shown on the computer screen then can be fine-tuned for product design using standard computer-aided design/modeling software.

"This technology will give product designers, or even artists, a tool that will allow them to touch, shape and manipulate virtual objects just as they would with actual clay models or sculptures," says Thenkurussi Kesavadas, director of the Virtual Reality Lab and associate professor of mechanical and aerospace engineering in the School of Engineering and Applied Sciences.

Computing Gets Physical

Wed, 30 Jun 2004 09:39:47 -0400

Gadgets that let you control computers with a wave or a nod could offer an escape from keyboards and mice.

For once, I control the weather.

I’m standing in front of a green backdrop inside a windowless studio at Cybernet Systems, a technology research and development company in Ann Arbor, MI. A digital camera in front of me is beaming my image, real time, to a television monitor that shows a scene typical of a nightly news weather report. There I am, standing before a map of the Midwest. I extend my arm and begin twirling my hand over the blip of Detroit. The map behind me zooms in on the area beneath my palm. The city widens into view and comes into focus. Looks like it’s going to be a wet one, folks.

This is GestureStorm—a software system Cybernet developed to let weather broadcasters run through their forecasts with simple flicks of the hand. No wires. No buttons. No geeky audiovisual control panels. Move a hand one way, and you paint raindrops on-screen. Move it another, and you stir up a tornado. The interface is completely a matter of gesture. And if a lot of people have their way, this is only the beginning. Gesture recognition technology aims to become this millennium’s remote control—a fluid, freeing means of interacting with all the digital stuff around us. Think Minority Report. In that film, Tom Cruise stands before a futuristic digital display, pointing and waving his way through a cascade of images and documents. This stuff, once the domain of science fiction, is finally creeping into the real world.

Fragments boost 3D TV

Wed, 23 Jun 2004 15:58:33 -0400

In January 2001, CBS spiced up its coverage of the Super Bowl with a special effect that allowed the broadcaster to freeze a replay, arbitrarily change the viewpoint and continue the replay. Researchers around the world are looking to take this technology further by enabling viewpoint changes as the action, including live-action, unfolds, and by letting viewers controlled viewpoint.

The formidable technical challenge in presenting real-time, free-viewpoint three-dimensional video is the enormous amount of information contained in the stream of video information.

Researchers from Swiss Federal Institute of Technology (ETH) in Zürich have devised a way to process three-dimensional video in real-time that reduces the amount of data to the manageable level of 3 megabits per second.

Organic LED Displays (OLEDs) - The Next Trend?

Tue, 01 Jun 2004 13:37:10 -0400

Wouldn’t you like to be able to read off the screen of your laptop in direct sunlight? Your mobile phone battery to last much, much longer? Or your next flat screen TV to be less expensive, much flatter, and even flexible? Thanks to a breakthrough technology called Organic Displays, this could soon be reality.

Although the technology behind Organic LED (OLED) displays is pure chemistry, the applications are much more everyday - mobile telephone and television screens, laptop and stereo displays, car navigation systems, or even billboards.

"Aristotle" (The Knowledge Web)

Mon, 31 May 2004 00:20:15 -0400

(DANNY HILLIS:) I have always envied Alexander the Great, because he had Aristotle as a personal tutor. In those days, Aristotle knew pretty much everything there was to know. Even better, Aristotle understood the mind of Alexander. He understood which topics interested Alexander, what Alexander knew and did not know, and what kinds of explanations Alexander preferred. Aristotle had been a student of Plato, and he was himself a great teacher. We know from his writings that he was full of examples, explanations, arguments, and stories. Through Aristotle, Alexander had the knowledge of the world at his command.

Of course no one today knows all that is known, in the sense that Aristotle did. Now there is far too much knowledge for that to be possible. The scientific revolution, and the technological revolution that followed it, led to a self-reinforcing explosion of knowledge. The explosion continues. Today not even the most highly trained scientist, the most scholarly historian, or the most competent engineer can hope to have more than a general overview of what is known. Only specialists understand most of the new discoveries in science, and even the specialists have trouble keeping up.

This problem isn't new. In 1945, Vannevar Bush wrote an essay for Atlantic Monthly about out the problem of too much knowledge. He wrote,

Brain chips could help paralysed

Mon, 19 Apr 2004 14:32:05 -0400

Scientists are to implant tiny computer chips in the brains of paralysed patients which could 'read their thoughts'.

US researchers from Cyberkinetics Inc are to be allowed to implant the chips underneath the skulls of patients. The chips will map the neural activity which occurs when someone thinks about moving a limb. Scientists will then translate those signals into computer code that could one day be fed into robotic limbs.

The company, based in Foxboro, Massachusetts, has been given Food and Drug Administration approval to begin the trials of the four-millimetre square chips.

The 'Brain Gate' contains tiny spikes that will extend down about one millimetre into the brain after being implanted beneath the skull, monitoring the activity from a small group of neurons. The signals will be monitored through wires emerging from the skull, which presents some danger of infection. The company is working on a wireless version.