The Math of Confused Eyewitnesses

"Yes, he's definitely the one I saw that night. I'll never forget that sneer."

Confident, but mistaken eyewitness reports during criminal trials can send an innocent man or woman to prison. And new experiments (as well as common sense) indicate that such faulty identifications of suspects are not uncommon.

Before I get to developments on this topic, consider a coin puzzle whose solution is relevant to the issue.

Picking the Biased Penny

Assume that you have three suspect pennies lined up before you. You're told that one of these pennies, the culprit, lands heads 75 percent of the time, and that the other two, the innocent suspects, are fair coins. You know nothing else about the pennies, but you did previously observe that one of the coins was flipped three times and landed heads all three times.

Having witnessed this and realizing that the biased penny is much more likely to behave in this way, you identify this coin as the culprit.

How likely are you to be right?

If you were randomly to pick a penny from the lineup, the probability that it would be the culprit would be 1/3 or about 33 percent. But given that you have this (less than conclusive) information about one of the pennies, what is the probability that it is the culprit?

The answer to the problem, obtained using what is known as Bayes Theorem (see the sidebar below if you must know how), is 63 percent. We revise our probability estimate of that penny's being the culprit upward from 33 percent to 63 percent because it's been flipped three times and has landed heads all three times.

The calculations are formally analogous to what we do when we change our estimate of the probability of a suspect's guilt after the testimony of an eyewitness. Identifying a biased coin on the basis of the evidence of three consecutive heads is mathematically the same as identifying a human culprit on the basis of an eyewitness' memory.

Picking the Wrong Guy

There are, of course, many complicating issues in the case of eyewitnesses and suspect lineups. An article by Atul Gawande in the Jan. 8 edition of The New Yorker details the work of Gary Wells, a psychologist at Iowa State University, and others who have noted the alarming error rate among eyewitnesses to crimes. They have discovered a number of factors that significantly influence the likelihood that witnesses will correctly pick the culprit out of a lineup.

Is the lineup of suspects simultaneous or sequential? (Sequential presentation is usually much better.) Do the authorities make an effort to have the others in the lineup match the culprit's description? (More reliable identifications are generally obtained this way.) Is the eyewitness told that the culprit may or may not be in the lineup? (If he's not, he's more likely to pick the person who most resembles the culprit.)

Professor Wells has also devised experiments to determine how often eyewitnesses pick the wrong man out of a lineup, and the results are terrifying.

Despite the fact that eyewitnesses are usually quite certain of who and what they've seen, the probability of a correct identification (after people have seen videotape of a simulated crime, for example) is frequently as low as 60 percent, and, what's worse, innocents in the lineup are picked up to 20 percent or more of the time, percentages not much better than in the penny example.

This is not a trivial problem, since it's estimated that almost 80,000 people annually become criminal defendants after being picked out of a lineup by eyewitnesses.

Calculating the Accuracy of a Witness

Using Bayes' Theorem, Wells points out that the base rate likelihood — the initial probability that a suspect is the culprit — greatly affects the subsequent likelihood — the probability that he is the culprit given that he has been picked out of a lineup. If the base rate likelihood is small, eyewitness' identifications need to be almost certain to ensure the subsequent probability is reasonably high. In the penny problem discussed above, the base rate likelihood of picking the culprit is 33 percent, the subsequent likelihood 63 percent.

Another useful notion that Wells explores is the information gain from eyewitness identification. This is the difference between the subsequent likelihood that he is the culprit given he's been identified as such and the base rate likelihood that he is.

Again considering the biased penny to be a culprit of sorts, we conclude the information gain from seeing a penny flipped three times and landing heads all three times is the difference between the probability the penny is the culprit given that it's landed heads three times (63 percent) and the initial probability the penny is the culprit (33 percent). The information gain is thus 30 percent.

There are, of course, many other nuances and variations but the bottom line is that the eyewitness testimony on which people are convicted is sometimes not worth three cents.

Professor of mathematics at Temple University, John Allen Paulos is the author of several best-selling books, including Innumeracy and A Mathematician Reads the Newspaper. His Who’s Counting? column on appears on the first day of every month.