Education statistics are a hot topic and, as usual, they are being spun left and right.
Even simple notions like averages can lead to trouble, with odd things happening if one carelessly averages averages.
My topic for this month concerns two such odd results — one old but still surprising, the other just published. Warning: Both cause headaches.
Sex Discrimination Paradox
First the old one, which is often called Simpson's paradox (no, not that Simpson). A sex discrimination case in California a while ago has become a sort of classic illustration. Looking at the proportion of women admitted to the graduate school at the University of California, some women sued the university claiming they were being discriminated against by the graduate school. When administrators looked for which departments were most guilty, however, they were a little astonished to find that there was actually a positive bias for women.
To keep things simple, let's suppose there were only two departments in the graduate school, economics and psychology. Making up numbers, let's further assume that 70 of 100 men (70 percent) who applied to the economics department were admitted and that 15 of 20 women (75 percent) were. Assume also that five out of 20 men (25 percent) who applied to the psychology department were admitted and 35 of 100 women (35 percent) were. Note that in each department a higher percentage of women was admitted.
If we amalgamate the numbers, however, we see what prompted the lawsuit: 75 of the 120 male applicants (62.5 percent) were admitted to the graduate school as a whole whereas only 50 of the 120 female applicants (41.7 percent) were.
Such odd results can surface in a variety of contexts. For example, a certain medication X may have a higher success rate than another medication Y in several different studies and yet medication Y may have a higher overall success rate. Or a baseball player may have a lower batting average than another player against left-handed pitchers and also have a lower batting average than the other player against right-handed pitchers but have a higher overall batting average than the other player.
Of course, these counter-intuitive results don't usually occur, but they do often enough for us to be wary of uncritically combining numbers and research studies into a so-called meta-analysis.
Course Load Paradox
A new oddity that has a similar flavor was published in the spring issue of Chance Magazine (http://www.public.iastate.edu/~chance99/) by Randy Mason, and it concerns a paradox in a common way of presenting marketing statistics (although I'll state it here in terms of college courses taken). Involving overlapping groups of people, the paradox is best understood via two tables.
The first table shows the outcomes of a survey of eight students who were asked about the number and type of college courses they took during the previous year.
The second table gives the average number of courses taken by the eight people and the average number of courses taken by those who took specific types of course.
A possible problem arises when we note that the average number of courses taken by all eight people is smaller than the average number of courses taken by those who took at least one course of a particular type. That is, if we look only at the three people who took at least one math course, we find that the average number of courses they took (8.33) is bigger than the average number of courses taken by all eight people (5.75).
OK, you say, maybe there's something special about those people who take math courses that explains why they take more courses on average.
But if we look only at the four people who took at least one science course, we find that the average number of courses they took is also bigger (8.25) than the average number of courses taken by all eight people. Again, we might think there's something special about those people who take science courses that explains why they take more courses on average.
The full paradox appears after we finish checking those who took at least one social science or one humanities course and discover that the same phenomenon holds for them. If we look at those who took at least one course of type X, whatever X is, we will find that the average number of courses taken by this group is higher than the average number reported by the whole group.
It seems that using this particular method of reporting people's course loads can sometimes justify a Lake Wobegon-style boast that those who take any particular type of course take an above average number of courses.
This odd result may be of use to spin doctors in politics or education.
Professor of mathematics at Temple University, John Allen Paulos is the author of several best-selling books, including Innumeracy and A Mathematician Reads the Newspaper. His Who’s Counting? column on ABCNEWS.com appears on the first day of every month.