A Computer That Can Read Your Mind

July 7, 2008 — -- One of these days your computer will probably know what you are thinking before you know it yourself. The human face conveys emotions ranging from fear to confusion to lying, sometimes involuntarily, and scientists are figuring out how to make use of those expressions.

At the University of California at San Diego, for example, a graduate student has developed a program that will slow down or speed up a video based entirely on changes in his facial expressions, like a slight frown, or a smile. The purpose of this particular program is to make robotic instructors more responsive to their student's needs, but there are many other potential applications for the work.

"The project I'm working on is how can we use machine perception, including things like facial expressions, to improve interactivity between students and teachers," said Jacob Whitehill, a computer science doctoral candidate. "That includes human teachers, and also robotic teachers, which is something our lab is increasingly interested in."

Whitehill has tested his technology on several other students, using software developed in the university's machine perception laboratory, called the Computer Expression Recognition Toolbox, or CERT. The machine got it right nearly half the time, slowing down a video of a lecture if the student was having trouble following it, and speeding it up if the student was swimming along.

"While these results show room for improvement, they already demonstrate the value of automatic real-time facial expression recognition in intelligent tutoring systems," Whitehill said in a paper describing his research.

In an interview he described how the system works.

A student sits in front of a computer as a webcam captures images of the student's face. The images are fed through a series of filters to keep useful information and remove useless information. The machine has already been programmed to recognize certain expressions, "like a frown, or a smile or a nose wrinkle."

When it worked right, the video slowed down if the student showed signs of confusion, and sped up if the expression indicated that the lecture was being absorbed.