If the Dunce Cap Fits, Wear it

The latest Newsweek poll, purporting to show how dim most Americans are, puts the dunce cap on the wrong head.

The magazine is following a well-worn path – sneaking up on a bunch of well-intentioned survey respondents, springing a bunch of unrelated factual questions on them and then gleefully reporting that they don’t know jack. As a cheap polling trick, it works every time. As an exercise in meaningful measurement, it makes Paris Hilton look sage.

The first problem here is the all-too-common confusion of knowledge with recall. At best, recall is all that Newsweek measures – the ability to recite disassociated facts at the drop of a hat. It asks, in order, Who wrote "Pride and Prejudice?" Who won American Idol this year? What's the world's most popular spectator sport? And so on – 29 questions in all, jumping around from topic to topic like Ken Jennings on too much caffeine.

This has decidedly nothing to do with knowledge. Knowledge reflects the ability to think – not merely to recite information, but to use it to draw connections and build concepts. You can have deep knowledge, but weak on-command recall. You can have terrific recall, but little knowledge. Life is not a game show.

Imagine, for example, that I ask you to recite the 10 Commandments. Maybe you can. But if not, does it mean, de facto, that you’re not a religious person, that you don’t comprehend the basic tenets of the Judeo-Christian ethic, and that therefore you'll fry in hell with nothing to read but tattered dentist-office copies of a certain national newsweekly magazine? Maybe not.

The measurement, moreover, is as faulty as the concept. Opinion polls are called opinion polls for good reason. They're excellent at measuring opinion. They also happen to be pretty good at measuring behavior. But they're poor tools for measuring recall, much less knowledge.

The reasons are simple. Think of memory as a stack of file cards. Attitudes (and most behaviors) are right there toward the top – frequently used, thus easily accessed. Facts, by contrast, are buried deeper. (When's the last time you pondered Jane Austen?) A person participating in an opinion poll will gaily give you their opinions. Digging out facts requires cognitive work of a completely different order – and with the spaghetti on the stove and two kids squalling in the other room, it's far less likely to fly.

Facts and knowledge carry two other qualities that bedevil their easy measurement. One is that they're memory-activated; engage someone in a thoughtful conversation about the Middle East and they'll be able to remember and conceptualize more about it. Another is that facts are freighted with the weight of error. Opinions are risk-free; they can’t be right or wrong. Ask people a factual question and they’re more likely to beg off to avoid the embarrassment of a bad answer. Hence Newsweek tells us that 81 percent don’t know the name of the Supreme Court's chief justice. For some that's surely so. (And what of it?) For others, it more likely means that they just didn’t want to play this silly game.

Nor is it any wonder that more people can name Jordin Sparks than John Roberts. Sparks is a current recipient of the glare of popular culture; that she's simply better-known does not remotely suggest that her presence is more profound than anyone else's – Roberts most certainly included.

Yet another problem with Newsweek's approach is the unreality of some of its questions, such as one asking if the United States has captured Osama bin Laden. Some of the 11 percent who answered yes may well have been thinking, "OK, if you’re goofing with me, I'll goof with you." Not a helpful dynamic to create with 21 more questions yet to go. (Price of oil? Chairman of the Fed? Number of nuclear-armed nations?)

Imagine another approach: Say we ask our respondents to come to a testing center where we engage them in discussion of an issue of the day. Then we hand them sharpened No. 2 pencils and ask them to fill out answer sheets testing both facts and knowledge relating to that issue – with the promise, for good measure, of $10 for every right answer, and $20 for every well-argued concept. I'll bet that simple change in measurement would make our "dunce-cap nation," as Newsweek would have it, suddenly look a lot sharper.