School's getting out, but the dreaded Scholastic Assessment Test, better known as the SAT, looms just a summer away for next year's high school seniors.
Given this, many might be inclined to agree with the president of the University of California, who announced several months ago that he would like to abolish the test as a requirement for admission to the school. (He would retain the SAT II, which measures achievement within particular disciplines.)
The announcement of his intention sparked an ongoing controversy.
The issue is complicated, and any full discussion should address the issue of the change in scores over the years. (I wish I had a dollar for every baby boomer I've heard say that the scores started to decline just after they graduated from high school).
Other important issues are the renorming of the test, culturally biased questions, ethnic and gender differences in the scores, self-selection of test-takers, differential participation of various sub-groups, the importance of calculators, test preparation, etc.
Artificially Low Correlation?
The big question, however, is: How predictive of success in college are SAT scores? More precisely, what is the correlation between high school SAT scores and first-year college grade point average? (The appropriateness of GPA as a measure of success is also open to question. Grades, for example, often depend critically on the courses taken.)
Most studies find that the correlation between SAT scores and first-year college grades is not overwhelming, and that only 10 percent to 20 percent of the variation in first-year GPA is explained by SAT scores.
This association appears weaker than it is, however, for an interesting, but seldom noted statistical reason: Colleges usually accept students from a fairly narrow swath of the SAT spectrum.
The SAT scores of students at elite schools, say, are considerably higher, on average, than those of students at community colleges, yet both sets of students probably have similar college grade distributions at their respective institutions.
If both sets of students were admitted to elite schools or both sets attended community colleges, there would be a considerably stronger correlation between SATs and college grades at these schools.
Those schools that attract students with a wide range of SAT scores generally have higher correlations between the scores and first-year grades.
This is a general phenomenon; the degree of correlation between two variables depends on the range of the variables considered.
Soccer Assessment Test (SAT)
The SAT traditionally deals with analogies, so let's consider soccer leagues.
Assume there were an SAT (Soccer Assessment Test) that measured the speed, coordination, strength, and soccer experience of students in a certain city. Assume further that the students roughly divided themselves into five leagues depending on their scores on this SAT, players in the top leagues having higher SAT scores on average than those in the lower leagues.
One wouldn't expect that a measure of success in the sport, say number of goals scored, to vary much among the leagues. There would be good scorers and bad scorers in every league and, just as grade point distributions are similar in most colleges, the distribution of goals scored would probably be similar in the five leagues.