Steven Pinker: Standardized Testing and Random Selection for College Admissions

New Republic

Some excerpts from Steven Pinker's (Harvard University, Department of Psychology) refreshing article. First he recaps the new book, Excellent Sheep, and offers a refutation of it (one that is based on personal anecdote, by the way, not data).

It’s not surprising that William Deresiewicz’s “Don’t Send Your Kid to the Ivy League” has touched a nerve. Admission to the Ivies is increasingly seen as the bottleneck to a pipeline that feeds a trickle of young adults into the remaining lucrative sectors of our financialized, winner-take-all economy. And their capricious and opaque criteria have set off an arms race of credential mongering that is immiserating the teenagers and parents (in practice, mostly mothers) of the upper middle class.

Deresiewicz writes engagingly about the wacky ways of elite university admissions, and he deserves credit for opening a debate on policies which have been shrouded in Victorian daintiness and bureaucratic obfuscation. Unfortunately, his article is a poor foundation for diagnosing and treating the illness. Long on dogmatic assertion and short on objective analysis, the article is driven by a literarism which exalts bohemian authenticity over worldly success and analytical brainpower. And his grapeshot inflicts a lot of collateral damage while sparing the biggest pachyderms in the parlor. 

We can begin with his defamation of the students of elite universities. Like countless graybeards before him, Deresiewicz complains that the kids today are just no good: they are stunted, meek, empty, incurious zombies; faithful drudges; excellent sheep; and, in a flourish he uses twice, “out-of-touch, entitled little shits.” I have spent my career interacting with these students, and do not recognize the targets of this purple invective. Nor does Deresiewicz present any reason to believe that the 18-year-olds of today’s Ivies are more callow or unsure of their lives than the 18-year-olds of yesterday’s Ivies, the non-Ivies, or the country at large.

Then Pinker offers a wonderful précis of what one should learn during four years of liberal arts (and science) education, one that includes both content and process:

It seems to me that educated people should know something about the 13-billion-year prehistory of our species and the basic laws governing the physical and living world, including our bodies and brains. They should grasp the timeline of human history from the dawn of agriculture to the present. They should be exposed to the diversity of human cultures, and the major systems of belief and value with which they have made sense of their lives. They should know about the formative events in human history, including the blunders we can hope not to repeat. They should understand the principles behind democratic governance and the rule of law. They should know how to appreciate works of fiction and art as sources of aesthetic pleasure and as impetuses to reflect on the human condition. 

On top of this knowledge, a liberal education should make certain habits of rationality second nature. Educated people should be able to express complex ideas in clear writing and speech. They should appreciate that objective knowledge is a precious commodity, and know how to distinguish vetted fact from superstition, rumor, and unexamined conventional wisdom. They should know how to reason logically and statistically, avoiding the fallacies and biases to which the untutored human mind is vulnerable. They should think causally rather than magically, and know what it takes to distinguish causation from correlation and coincidence. They should be acutely aware of human fallibility, most notably their own, and appreciate that people who disagree with them are not stupid or evil. Accordingly, they should appreciate the value of trying to change minds by persuasion rather than intimidation or demagoguery.

I believe (and believe I can persuade you) that the more deeply a society cultivates this knowledge and mindset, the more it will flourish. The conviction that they are teachable gets me out of bed in the morning. Laying the foundations in just four years is a formidable challenge. If on top of all this, students want to build a self, they can do it on their own time.

He then criticizes Harvard's (and by implication other elite/selective colleges') "holistic" admissions procedures, which he suggests simply mask racial/ethnic/class/other biases.

At the admissions end, it’s common knowledge that Harvard selects at most 10 percent (some say 5 percent) of its students on the basis of academic merit. At an orientation session for new faculty, we were told that Harvard “wants to train the future leaders of the world, not the future academics of the world,” and that “We want to read about our student in Newsweek 20 years hence” (prompting the woman next to me to mutter, “Like the Unabomer”). The rest are selected “holistically,” based also on participation in athletics, the arts, charity, activism, travel, and, we inferred (Not in front of the children!), race, donations, and legacy status (since anything can be hidden behind the holistic fig leaf). 

The lucky students who squeeze through this murky bottleneck find themselves in an institution that is single-mindedly and expensively dedicated to the pursuit of knowledge. It has an astonishing library system that pays through the nose for rare manuscripts, obscure tomes, and extortionately priced journals; exotic laboratories at the frontiers of neuroscience, regenerative medicine, cosmology, and other thrilling pursuits; and a professoriate with erudition in an astonishing range of topics, including many celebrity teachers and academic rock stars. The benefits of matching this intellectual empyrean with the world’s smartest students are obvious. So why should an ability to play the bassoon or chuck a lacrosse ball be given any weight in the selection process? 

The answer, ironically enough, makes the admissocrats and Deresiewicz strange bedfellows: the fear of selecting a class of zombies, sheep, and grinds. But as with much in the Ivies’ admission policies, little thought has given to the consequences of acting on this assumption. Jerome Karabel has unearthed a damning paper trail showing that in the first half of the twentieth century, holistic admissions were explicitly engineered to cap the number of Jewish students. Ron Unz, in an exposé even more scathing than Deresiewicz’s, has assembled impressive circumstantial evidence that the same thing is happening today with Asians.

Just as troublingly, why are elite universities, of all institutions, perpetuating the destructive stereotype that smart people are one-dimensional dweebs? It would be an occasion for hilarity if anyone suggested that Harvard pick its graduate students, faculty, or president for their prowess in athletics or music, yet these people are certainly no shallower than our undergraduates. In any case, the stereotype is provably false. Camilla Benbow and David Lubinski have tracked a large sample of precocious teenagers identified solely by high performance on the SAT, and found that when they grew up, they not only excelled in academia, technology, medicine, and business, but won outsize recognition for their novels, plays, poems, paintings, sculptures, and productions in dance, music, and theater. .. 

 

And he suggests a solution -- the SAT! Admit 10% of the class based on their top scores, then admit the rest by lottery, given that they exceeded some pre-stated score threshold (i.e., that they demonstrated they that are smart enough to benefit from a Harvard education).

 ...If only we had some way to divine the suitability of a student for an elite education, without ethnic bias, undeserved advantages to the wealthy, or pointless gaming of the system. If only we had some way to match jobs with candidates that was not distorted by the halo of prestige. A sample of behavior that could be gathered quickly and cheaply, assessed objectively, and double-checked for its ability to predict the qualities we value….

We do have this magic measuring stick, of course: it’s called standardized testing. I suspect that a major reason we slid into this madness and can’t seem to figure out how to get out of it is that the American intelligentsia has lost the ability to think straight about objective tests. After all, if the Ivies admitted the highest scoring kids at one end, and companies hired the highest scoring graduates across all universities at the other (with tests that tap knowledge and skill as well as aptitude), many of the perversities of the current system would vanish overnight. Other industrialized countries, lacking our squeamishness about testing, pick their elite students this way, as do our firms in high technology. And as Adrian Wooldridge pointed out in these pages two decades ago, test-based selection used to be the enlightened policy among liberals and progressives, since it can level a hereditary caste system by favoring the Jenny Cavilleris (poor and smart) over the Oliver Barretts (rich and stupid).

If, for various reasons, a university didn’t want a freshman class composed solely of scary-smart kids, there are simple ways to shake up the mixture. Unz suggests that Ivies fill a certain fraction of the incoming class with the highest-scoring applicants, and select the remainder from among the qualified applicant pool by lottery. One can imagine various numerical tweaks, including ones that pull up the number of minorities or legacies to the extent that those goals can be publicly justified. Grades or class rank could also be folded into the calculation. Details aside, it’s hard to see how a simple, transparent, and objective formula would be worse than the eye-of-newt-wing-of-bat mysticism that jerks teenagers and their moms around and conceals unknown mischief. 

Pinker blasts the "standardized tests don't predict anything useful" nonsense, as well as the "SAT scores are correlated with parental income, so therefore they are biased" nonsense:

So why aren’t creative alternatives like this even on the table? A major reason is that popular writers like Stephen Jay Gould and Malcolm Gladwell, pushing a leftist or heart-above-head egalitarianism, have poisoned their readers against aptitude testing. They have insisted that the tests don’t predict anything, or that they do but only up to a limited point on the scale, or that they do but only because affluent parents can goose their children’s scores by buying them test-prep courses. 

But all of these hypotheses have been empirically refuted. We have already seen that test scores, as far up the upper tail as you can go, predict a vast range of intellectual, practical, and artistic accomplishments. They’re not perfect, but intuitive judgments based on interviews and other subjective impressions have been shown to be far worse. Test preparation courses, notwithstanding their hard-sell ads, increase scores by a trifling seventh of a standard deviation (with most of the gains in the math component). As for Deresiewicz’s pronouncement that “SAT is supposed to measure aptitude, but what it actually measures is parental income, which it tracks quite closely,” this is bad social science. SAT correlates with parental income (more relevantly, socioeconomic status or SES), but that doesn’t mean it measures it; the correlation could simply mean that smarter parents have smarter kids who get higher SAT scores, and that smarter parents have more intellectually demanding and thus higher-paying jobs. Fortunately, SAT doesn’t track SES all that closely (only about 0.25 on a scale from -1 to 1), and this opens the statistical door to see what it really does measure. The answer is: aptitude. Paul Sackett and his collaborators have shown that SAT scores predict future university grades, holding all else constant, whereas parental SES does not. Matt McGue has shown, moreover, that adolescents’ test scores track the SES only of their biological parents, not (for adopted kids) of their adoptive parents, suggesting that the tracking reflects shared genes, not economic privilege. 

In general, I agree with Pinker. The resume-stuffing, 15 AP classes in high school, varsity captain, first violin, community service nonsense in college admissions must stop. He's right that college isn't about pulling an oar or playing a bassoon.

I would advocate for a beefed up SAT (too many students get "perfect scores," so there's a ceiling effect), perhaps one that incorporated the Miller Analogies Test as well as the multiple choice sections from an assortment of AP exams.

I would hope that a consortium of selective colleges get together and all use the same admission test. If you score above a preordained threshold (say, 90% percentile), then you are randomly assigned to one of the 20 colleges in the consortium (e.g., Harvard, Duke, Stanford, etc.). With random assignment, each of these colleges are guaranteed to get their fair share of talented athletes, musicians, poets, etc. They will also have the same percentages of females and people from minority groups. They will also have the same percentage of billionaires' kids. All of the students will be smart enough to benefit from the educational opportunities available. Any differences in "success" among the various schools' graduates will be attributable to differences in their educational programs, and not to selection effects. Best of all, the focus will switch from "getting in" to doing well while you are in college. People shouldn't feel special just because they got into Harvard. Wouldn't that change if they knew that the reason they got in wasn't because some admissions committee thought that they were super-special, but because they were just plain lucky?

My guess is that this will never happen, and for the same reason that you don't go to a jeweler and say, "I want to spend $10,000 on a watch" only to have them tell you: "Great. I'll take your money and reach into this bag and pluck you out either a Rolex, or an Omega, or a whatever." People think that the brand name matters, and that's what they're paying for, not the education. Harvard (and other selective colleges) are selling a brand name luxury product, not an education.