Study: Computers Can Now Predict What You're Seeing

March 6, 2008— -- If you're paranoid, you might want to look the other way — according to a new study, someone hoping to peer into your thoughts may one day have an interesting tool to help them do just that.

Researchers at the University of California at Berkeley have developed a computer program that can predict what a person is looking at by analyzing his or her brain activity.

In a letter published in the journal Nature, Jack L. Gallant and his colleagues report their computer program correctly identified which single picture out of 120 photos a person was looking at 72 to 92 percent of the time.

In the same circumstances, a random guess would only be correct 0.8 percent of the time.

But rest assured, drive-by mind reading is a long way off for now. In order to predict what someone is looking at, the computer program has to get to know the person.

"Your brain looks different from my brain — at the fine scale it looks totally different," said Frank Tong, professor and researcher at Vanderbilt University in Nashville, Tenn., who was not associated with the study.

How It Works

The researchers described their study as "analogous to the classic 'pick a card, any card' magic trick," except that each card is a unique photo.

First the researchers showed test subjects hundreds of stock photos of natural objects — trees, horses, fruits. Meanwhile the researchers simultaneously measured the person's brain activity with a brain scan called functional magnetic resonance imaging, or fMRI.

The researchers then used computer programs to analyze the differences between the brain scan of a person when he was looking at, say, a photo of a horse versus a photo of an orange.

When a person was tested again with a brand new set of photos, the computer program could guess which of the photos the person was now looking at based on their brain activity when they were looking at the other photos.

"The more they [could] measure your brain activity — for hours on end for months and months — they would get a better and better at predicting what you're looking at," Tong said.

"I think this experiment is a demonstration that this can be done, and it can be done pretty darn well," he said.

However, just because something can be done doesn't mean that people will believe it should be done. Historically, devices like polygraphs have spurred controversy over how deeply into another person's mind we should be allowed to peer.

Should it Work?

In their paper, the study's authors indicated they thought their latest computer program could be a building block for more intriguing mind-reading capabilities to study where someone is focusing, or read the visual content of dreams.

"Our results suggest that it may soon be possible to reconstruct a picture of a person's visual experience from measurements of brain activity alone," the authors wrote in their paper.

Tong, on the other hand, believes this advance will more likely be a gateway to understand how our brains process visual information before it develops into a more robust mind-reading tool.

"You could have a multimillion dollar machine to read someone's brain to see what they're looking at — or, you can do the cheap method and look over their shoulder," he said.

A harder question, Tong believes, would be to find a computer model to explain why humans can infer that the red and purple colors in the sky add up to a sunset, or the black and white colors of fur add up to a dog.

"If we could have cameras that could recognize things, we could have robots that could drive cars, robots that could do surgery," Tong said.