How to Write on Thin Air

June 17, 2009 — -- Wouldn't it be handy if you could write yourself a note in midair and have it safely stored where you could retrieve it later? Someday soon you may be able to do just that with nothing more than your cell phone.

Researchers at Duke University have created what they are calling a "PhonePoint Pen" that uses a modern cell phone's built-in accelerometer to detect writing in thin air, and then automatically send a note to a designated e-mail account. There are still some bugs to work out, but the researchers believe they've proven that the idea really works.

"We are convinced that this is feasible and this will become something that people will use," Romit Roy Choudhury, assistant professor of electrical and computer engineering at Duke, said in a telephone interview.

Roy Choudhury said he began toying with the idea while he was still a grad student in 2005, back in the days of technological antiquity when the most common form of writing a personal reminder was with a sticky note.

"I used to keep forgetting things, and I thought there should be some way of jotting something down while I'm walking down the street, like calling someone when I get home," Roy Choudhury said. "By the time I get home, I've forgotten about it."

Accelerometer Detects Movement of Pen

"The existing technology wasn't all that great," he added. "And the idea struck me that maybe it's possible that I could have a pen with an accelerometer and I could just write in the air with the pen."

The accelerometer would detect the movement of the pen, and "then and I could press a button or something and the writing would get e-mailed to my mailbox. But getting a pen with an accelerometer was hard."

Fast forward three years and Roy Choudhury is an assistant professor of electrical and computer engineering at Duke. A lot had happened during that brief period, including the inclusion of accelerometers in state-of-the-art cell phones.

That's the feature that allows an image on the screen of an iPhone to change between portrait and landscape formats as the phone is rotated.

Cell Phone as the Magic Pen

As an assistant professor, Roy Choudhury had a gaggle of creative students at his disposal, and he suggested they try to figure out how to use a cell phone as the magic pen he had wanted in college.

At first, it was kind of a game, he admits, but Sandip Agrawal, an electrical and computer engineering senior, and Ionut Constandache, a grad student, made rapid progress.

One cell phone manufacturer, Nokia, donated "a bunch" of phones, and the students found that if they held the phone like a pen, grasped between the thumb and the forefinger, they could control it's movements well enough for it to recognize letters of the alphabet.

They had to be pretty big letters, about six inches tall, and the user had to learn how to write with no frame of reference other than an imaginary blackboard, but it worked well enough to jot down a phone number, or an address, or where the car was parked at the airport.

Phone Can Recognize Script

That's progress, but it's not good enough. The researchers now say they've figured out how to make the phone recognize script, and translate that into text before sending it to a designated e-mail account.

"The phone can track what you are writing," Roy Choudhury said, and even if your penmanship isn't perfect it should be able to figure out which letter of the alphabet you are trying to write. Meanwhile, the accelerometer will track other movements as well. It should be possible to "write" yourself a note while driving your car.

"There are a lot of signals that the accelerometer captures," he added. "From all those signals, we can pluck out the part that comes from the moving vehicle, because there is a particular signature from a car's movements, and we can subtract that. "

A moving hand, for example, can change direction much more frequently and more quickly, than a vehicle. So the car can be taken out of the equation.

That may not sound too safe, but it could be a lot better than trying to text-message yourself on a keyboard that just seems to get smaller and smaller.

Real-Time Alerts on Traffic Congestion Via Cell Phone

The device would also be able to clean up the signal while the user is walking, or doing just about anything else, he added.

The Duke researchers are also working on an idea that has intrigued many other scientists since the advent of cell phones with accelerometers. Theoretically, it should be possible for cell phones to provide real-time alerts on traffic congestion.

"Imagine you want to go to the airport and you want to know what the traffic on I-40 is like," Roy Choudhury said. "You could potentially get data from the accelerometer readings of all the phones that are traveling on I-40 at that moment."

The data could be captured by the cell phone network and fed into a computer that could plot traffic conditions over a wide area as events are unfolding. So before heading for the airport, check the Web to see which route would be the quickest at that precise moment.

These new uses for cell phones would not require changes to the phones themselves, he added. The changes would be in the software, not the hardware, and because accelerometers are always on, there would be no additional drain on the batteries.

There are still "issues" to be worked out, Roy Choudhury said, but his students have some tangible evidence that they aren't just toying around. They traveled to Chicago earlier this month to pick up the first Hoffman+Krippner Award for Excellence in Student Engineering.

Make a note of that.