Easy Call: 'Sign' Over Cell

Sept. 18, 2008 -- As a hearing child of deaf parents, Richard Ladner saw firsthand the impact of communications technology on his parents' lives.

"Back in the early 1970s, they got their first teletypewriter," he said. "It was a very big box, the size of a computer, but it opened a new world for them."

Now a professor of computer science and engineering at the University of Washington, Ladner sees another world opening with MobileASL, software he developed with six other engineers at his school and Cornell University. MobileASL allows deaf and hard-of-hearing people to "chat" over their cell phones in American Sign Language via two-way, real-time video.

"Speaking in sign language," Ladner said, "is a lot more natural -- and faster -- than texting," which is how deaf people communicate on cell phones. "Signing is like having an alternative that is a natural language."

Not that bringing ASL to a U.S. cell phone was an easy task: The low bandwidth on the standard U.S. cell phone network, coupled with the low processing power of most mobile phones, posed obstacles when it came to streaming video at rates fast enough to transmit intelligible sign language. In Europe and Asia, where higher bandwidth 3G networks predominate, cell phone signing is already under way in Sweden and Japan.

"The big challenge has been, how can we process enough frames per second on the cell phone in real time so we can actually have a video that people can understand," said Eve Riskin, a professor of electrical engineering at the University of Washington and the principal investigator on the project.

The team got around these barriers through video compression technology that selectively reduces the amount of data needed to show video images.

The key, though, lies in reducing data in the right areas.

In sign language, the hands -- and especially the face, which can convey subtleties in expressions, feelings, even grammar -- are the most important parts of the video image. Through "skin detection" algorithms that can pick out the hands and face in an image by matching blocks of pixels, the data constituting these important areas are transmitted in full for the highest resolution. For less important parts of the image, such as the background, the data packets can be dropped without sacrificing intelligibility.