My last post was not as solution-oriented as I generally try to be. I promised I’d talk about alternatives. I don’t have many answers, but I think I know some of the questions.
Some people ask: how can the system change? That, I think, is easy. By “the system” I just mean the fact of peer-reviewed publication working as the currency of academia. Killing a currency is as easy as killing Tinkerbell; just stop believing in it (as long as it’s not a tax-driven currency). If enough of us stop thinking that peer-reviewed publications have value, they stop having value.
A more important question is: what are the risks of devaluing the currency? I gave arguments in my last post that it wouldn’t stop us from accessing valuable ideas in philosophy–indeed, environments without peer review seem to allow much more interesting and original ideas to flourish. They might allow more bad ideas to flourish too, but so what? Find the flowers among the weeds, and maybe wiser minds will find something to do with the weeds one day (think of the “large gourd” in Zhuangzi 1.7).
Some of my colleagues pointed out another function currently played by the currency of peer-review, which I hadn’t considered. If we base hiring, promotion, tenure, and funding decisions on a candidate’s track record of peer-reviewed publications then that at least introduces a “blind” element to those decisions. Such decisions are commonly affected by the candidate’s gender, age, race, seniority, social class, and other prestige markers, such as where they did their PhD. Peer-review, if it’s genuinely blind, is not affected by these things. Thus peer-reviewed publication gives candidates who might otherwise have been blocked by bias a way to prove themselves. A graduate from a little-known university can build up a portfolio of impressive peer-reviewed publications, which might be enough to cut through the hiring panel muttering “State University of What?” or cringing at the wrong sort of accent.
This is a very valid concern, since any valuable improvement in academia has to push back the forces of elitism, snobbery, small-mindedness, and the club mentality. Anything that allows those to flow more freely is going in the wrong direction. So the idea that getting rid of peer review might do so is worrying to me.
Still, if this is the function of peer review, it’s a poor instrument. It has fallen victim to Goodhart’s Law: as soon as a measure becomes a target, it ceases to be a measure. If we imagine a world in which people publish articles only when they have ideas that they genuinely want to share with the world, and where people only review articles out of genuine interest, then you can imagine the number of somebody’s peer-reviewed publications in “top” journals being a measure of something useful. But once people know that getting anywhere in academia requires you to churn out several publications a year, they’ll publish in order to show that they can reliably pass peer review rather than passing peer review in order to publish. The tail wags the dog.
The result is that “top” journals are inundated with manuscripts. Reviewers, meanwhile, are no longer reviewing merely out of genuine interest. Some feel moral pressure to review articles, knowing that careers depend on it. Others want to block potential rivals from entering the scene. Others are motivated by the same sorts of lovely desires that drive people to join the border police. Some believe that reading an article is a prerequisite for reviewing it. Others are more flexible on this. Even if the majority of reviewers have responsible attitudes and noble motives, the sheer volume of manuscript submissions drives journals to scrape the barrel of reviewers, digging up the real rats and barnacles. Peer review therefore stops functioning as a measure of anything meaningful. All that passing peer-review tells you is that two reviewers eventually let you through, and God only knows what their reasons might have been. Gloria Origgi reports that peer-reviewed publications are already decreasing in perceived value, due to this Goodhart’s Law dynamic. And, needless to add, since the system now rewards those with a lot of time and resources to spend on writing and sending out manuscripts, its anti-elitist function is pretty enfeebled.
So how else might we counter bias in hiring? Universities, like most other corporations in the current climate, are obsessed with discrimination on obvious categories: gender, sexuality, race, disability, etc. That’s very important, of course, but if human biases worked on such obvious categories then problems of institutional inequality would be much easier to solve. A deeper sort of bias is a combination of status-quo and insider bias: hiring committees expect successful new candidates to look, sound, and act like successful old candidates. In a discipline like philosophy, this is bad news for the vast majority of human beings.
The advertising executive Rory Sutherland has an interesting take on this. He points out that there is a difference between the conscious and the unconscious motivations of hiring panels. Consciously, they’re just trying to find the best candidates. Unconsciously, they’re aware that choosing the wrong candidate can damage their reputation within the organisation. It is therefore important for them, not necessarily to choose the candidate they think will be best, but to make a choice they can justify to others. In practice, this means erring on the side of caution and convention.
Applying this insight in academic philosophy, everyone in your department will understand why you chose a candidate with a PhD from Oxford with 15 articles in “top” journals, each pretty much interchangeable with hundreds of others in the same journals. Hiring a Unicamp graduate with no publications, but who sent a writing sample sparkling with original insights, makes for a much more awkward departmental meeting. If both candidates turn out to be disappointing, the hiring panel will have to answer much more difficult questions about the second than the first. So, if they can only pick one, they’ll reliably pick the first.
Sutherland points out that the case might be different if organisations hire multiple people at once. Then the panel can have one “safety” candidate, to defend their reputation as sensible decision-makers, and go out on a limb with others. So one thing I’m in favour of is “bunching” hiring.
But will hiring panels (as well as tenure, promotion, and funding panels) be motivated to go out on a limb, provided they can protect their reputations? I choose to hope that they will, if we work hard to teach an important philosophical lesson. The philosophical lesson is this: you have no idea where the next great ideas will come from. What you do know is that they’re likely to come from voices that haven’t yet been heard, from perspectives that aren’t yet recognised. A computer scientist is reported to have said “a change in perspective is worth 80 IQ points”; the point is crudely put, but the principle seems sound. Being confronted with others who aren’t like us shakes our minds out of grooves and gets us to new places. If we want new ideas, we should widen our search for thinkers.
A series of purely random selections among candidates might bring up more gold, by covering a wider area in the search space, than continuing to dig where we already are. We’ve mined the perspective of Oxbridge and Ivy League graduates to the very tail of the seam. Of course venturing into uncharted territory means making lots of mistakes. The mistakes have short-term costs, and university administrators who can only read the right-hand columns of an annual report won’t like that one bit. But anyone who knows that it took 11 Apollos to get to the Moon will see through to the higher ideal. What we need is the intellectual humility and imagination to envisage good ideas coming from the last place we’d naturally think to look. Unfortunately, the system of peer review has drained a lot of intellectual humility and imagination from our discipline. It’s time to undrain the swamp.
Alexander Douglas 