Despite being profoundly deaf, 11-year-old Veronica and 8-year-old Zach are learning to speak, thanks in part to a computer-animated guide named Baldi.
Baldi is a revolutionary tutor whose 3-D lips, tongue and jaw movements are a near-perfect copy of human speech movements, known as phenomes.
"The big strength of Baldi," says Dr. Ron Cole, an expert in speech perception at the University of Colorado at Boulder, "is that it produces probably the most accurate automatic generation of visible speech in the world." This allows children to practice lip-reading.
[To take a look at Baldi, and to make 3-D characters speak what you program, see Web links at right.]
Learning to Speak
Working off the premise that spoken language is learned both by hearing words and seeing how to form them, Baldi helps deaf children learn how to form their words and practice pronunciation. They can mimic Baldi, and then, with Baldi's real-time feedback, find out if they've responded correctly.
To enhance Baldi's effectiveness, most of the kids wear headphones over an acoustic nerve implant. The device, inserted behind the ear during a three-hour operation, converts sound into electrical signals that can be relayed to the brain. Coupled with Baldi, say the staff at the Tucker-Maxon Oral School in Portland, Ore., deaf children's learning is revolutionized.
"It's easy to understand," says 13-year-old Chelsea Crump, who was born deaf. "Baldi talks clearly … so everybody can understand what he's saying."
Because Chelsea and her brother Timothy suffer from Ushers syndrome, a rare inherited condition that often results in progressive loss of vision and hearing, learning to speak with Baldi is even more critical.
The program has applications outside the deaf community. Children with autism, for example, who also have problems with verbal communication, are benefiting from Baldi. Cole believes the technology can be used to deal with an even more widespread problem.
"There are 20 million Americans that have reading disabilities," he says. "I'm absolutely convinced we could teach nearly every one of those children to read."
With technology like Baldi that incorporates speech recognition and speech synthesis, interactive books and interactive reading tutors could help kids sound out words, test comprehension and pronunciation, as well as offer new possibilities for creative expression.
"Anything's possible with 3-D real-time digital animation," says Eric Haseltine, executive vice president of research and development at the Walt Disney Co. (ABCNEWS.com's parent company), who also helps other companies develop such technology for multiple purposes.
"I think it will revolutionize human-computer interaction. Such characters will become the faces of the computer," he says. "People relate to characters and other people better than a computer, so this technology could bridge the gulf between computers and people."
As for day-to-day applications that extend beyond reading and speaking, Cole predicts such characters will be virtual office assistants within a few years. "You're talking about very natural conversational interactions with animated agents that will be your personal assistants and so forth," he says. So, for example, you might talk to the animated agent to make travel plans, and the 3-D persona might also respond with weather conditions.
Other applications might include speaking to the animated character at a fast-food drive-through or conducting a transaction at a bank machine with a 3-D agent that recognizes what you say and responds.
"Use your imagination," says Cole. "Whatever you think it can do, it can do."