Do SAT Scores Really Predict Success?
July 1, 2001 -- School's getting out, but the dreaded Scholastic Assessment Test, better known as the SAT, looms just a summer away for next year's high school seniors.
Given this, many might be inclined to agree with the president of the University of California, who announced several months ago that he would like to abolish the test as a requirement for admission to the school. (He would retain the SAT II, which measures achievement within particular disciplines.)
The announcement of his intention sparked an ongoing controversy.
The issue is complicated, and any full discussion should address the issue of the change in scores over the years. (I wish I had a dollar for every baby boomer I've heard say that the scores started to decline just after they graduated from high school).
Other important issues are the renorming of the test, culturally biased questions, ethnic and gender differences in the scores, self-selection of test-takers, differential participation of various sub-groups, the importance of calculators, test preparation, etc.
Artificially Low Correlation?
The big question, however, is: How predictive of success in college are SAT scores? More precisely, what is the correlation between high school SAT scores and first-year college grade point average? (The appropriateness of GPA as a measure of success is also open to question. Grades, for example, often depend critically on the courses taken.)
Most studies find that the correlation between SAT scores and first-year college grades is not overwhelming, and that only 10 percent to 20 percent of the variation in first-year GPA is explained by SAT scores.
This association appears weaker than it is, however, for an interesting, but seldom noted statistical reason: Colleges usually accept students from a fairly narrow swath of the SAT spectrum.
The SAT scores of students at elite schools, say, are considerably higher, on average, than those of students at community colleges, yet both sets of students probably have similar college grade distributions at their respective institutions.
If both sets of students were admitted to elite schools or both sets attended community colleges, there would be a considerably stronger correlation between SATs and college grades at these schools.
Those schools that attract students with a wide range of SAT scores generally have higher correlations between the scores and first-year grades.
This is a general phenomenon; the degree of correlation between two variables depends on the range of the variables considered.
Soccer Assessment Test (SAT)
The SAT traditionally deals with analogies, so let's consider soccer leagues.
Assume there were an SAT (Soccer Assessment Test) that measured the speed, coordination, strength, and soccer experience of students in a certain city. Assume further that the students roughly divided themselves into five leagues depending on their scores on this SAT, players in the top leagues having higher SAT scores on average than those in the lower leagues.
One wouldn't expect that a measure of success in the sport, say number of goals scored, to vary much among the leagues. There would be good scorers and bad scorers in every league and, just as grade point distributions are similar in most colleges, the distribution of goals scored would probably be similar in the five leagues.
In each league the better scorers would probably have only slightly higher SATs on average. In other words, there wouldn't be a high correlation between SAT scores and success in soccer within any league.
There would, however, be a much higher correlation between SAT scores and soccer success were the students randomly assigned to the teams in the five leagues.
(Similar remarks could be made about boxing, the different weight divisions washing out much of the correlation between greater weight and success at boxing.)
Of course, there are many dimensions of soccer ability that aren't measured by this imaginary SAT just as there are many, many dimensions of scholastic ability that aren't measured by the SAT. Concentrated work over an extended period is certainly one of the latter, the premium the SAT places on one morning's speedy work being especially difficult to defend.
The analogy between soccer and scholastics is not perfect, of course, but the point remains. Like the soccer "SAT," the scholastic SAT provides incomplete, but useful information to students and colleges. A rough measure of intellectual preparedness, the SAT shouldn't be made into a fetish, but neither should it be ignored.
Without it, colleges would undoubtedly place more emphasis on high school grades and extracurricular activities, measures that also have serious shortcomings — grade inflation and meaningless resume-puffing being the main ones.
The SAT is a flawed predictor, but it is also relatively objective and, among other virtues, sometimes provides a way for the bright, yet socially inept student to be recognized.
Professor of mathematics at Temple University, John Allen Paulos is the author of several best-selling books, including Innumeracy and A Mathematician Reads the Newspaper. His Who’s Counting? column on ABCNEWS.com appears on the first day of every month.