Publications–peer-reviewed publications in highly-ranked journals–are the currency of academic philosophy. This paper will address the following question:
WHY????? 😭😭😭😭😭😭😭😭😭😭😭😭😭😭
When I was getting into philosophy, I thought it was for creative, deep thinkers who wanted to get to the root of things and not take anything for granted. When I ask: what is the point of peer-reviewed publications in highly-ranked journals?, I expect philosophers to have answers–and thoughtful ones. Philosophy is the opposite of saying “I don’t know, that’s just what we do”. Philosophers ask why. Philosophers give reasons. What are our reasons?
We need reasons, after all, since we act as if we believe there to be some very important purpose served by these publications. We make enormous sacrifices to keep them pumping out. We make hiring and promotion decisions based on a candidate’s demonstrated aptitude for producing these publications. We allow vast amounts of university funding to be diverted into paying for subscriptions and licenses, which finance the companies or “charities” responsible for publication. Much of this money goes directly into dividends for shareholders in, e.g., RELX. We reward them for investing in something we value so much.
And there’s a lot of profit to divide, since the operating costs of a journal are quite low, especially if it’s all or mostly online. The creators of the “product”–the authors–write for free. Their remuneration comes in the form of improved chances of being hired/promoted. That’s thanks to us, of course. We’re on the hiring and promotion panels, so we create this incentive. We assist with the extraction of labour from authors. But we’re not greedy. We don’t keep the value extracted. That goes to the publisher, who certainly doesn’t give the “product” away for free. And who pays the publisher? Our employer: the university, which could otherwise have been paying that money to us.
We couldn’t be sending a clearer signal that we want the work but not the money. We are making a vast voluntary donation to a sector dominated by profit-driven firms. Voluntary collectively, that is: we all force each other to do it, but nobody forces us as a whole. We say: the hiring committees make us do it! The grant agencies make us do it! The research assessment panels make us do it! We are brutally oppressed by… ourselves–ourselves when we go to the next meeting and become the hiring committee, grant application reviewer, research assessment panel, etc.
What reason do we have for doing this? Is it, perhaps, legislated by the Categorical Imperative? Has there been a divine command? Or perhaps there is some great benefit provided by the academic publishers warranting our millions of hours of eleemosynary activity?
Well, what service do the publishers provide to the academic community? They make our research available to other scholars in our field. Wow! But wait. Tech enthusiasts might have heard of the Internet–or, as we young people call it, the ‘Net. It’s way cool, and you can get it for free at cafes and libraries. You can also get it at home for a low monthly cost. One of the features that makes the Net so totally radsticks is that you can upload your research there directly and link it to databases so that people interested in the topic can find it. If this Net thing takes off, I guess we don’t need the journals after all.
Some people will say: without journals, how would we know where to find quality research? Well, ask yourself honestly: of the important new stuff you read, how often do you read it in a published journal, and how often do you read a pre-print? In my case, it’s almost always a pre-print. The publishing process is much too slow for the development of ideas. But that means I almost always find the articles before they are published. How? Often authors get in touch with me to see if I’m interested in their work. I am, so they send it to me. Or I learn about it by word of mouth and use Google. We spend all this money and emit all this carbon travelling around to conferences, workshops, research visits–what is all that for if we can only find good research by browsing through journals? Journals made sense as a dissemination instrument when (a) the internet didn’t exist and (b) there were only a few journals for a given area and people would look at the contents list of each issue. I don’t know anyone who does that these days, and anyone who does is probably “not on the internet” and won’t reply to this post.
Aha, I hear you scream, but academic publishing provides another service: quality control. Academic publications select the very best research and bring it to the attention of everyone who reads all 190 ranked philosophy journals every quarter.
Well we’d better pay them for that service. I mean, you can’t expect to get a service like that for free–unless, of course, you’re an academic journal. They do get it for free, from… us. We provide the “quality control” in the form of blind peer review, also usually unpaid. We could, of course, just do this quality control directly on articles uploaded to the internet. Suppose, e.g., I want to read something good on Spinoza on hatred. Googling “Spinoza hate hatred”, I find a range of sources, published and unpublished. How do I know if they are any good? They haven’t been peer reviewed! Why, they could be any level of quality at all! At this point, I use a handy device called MY EYEBALLS. It’s called research–the thing that 40% of my salary is supposed to pay for. I trust my own judgement of quality better than Reviewer 2’s. And if I’m going to spend time reading something to determine its quality, I’d rather do that for myself, on a topic I’m researching, than for some journal in response to a review request.
So we don’t need journals to disseminate research. And we don’t need journals to judge the quality of research. We’re perfectly capable of doing these things ourselves. So what is the purpose of the journals? To certify research. Fundamentally, a journal is a brand. The brand’s reputation is maintained by peer reviewers (although what interest they have in preserving it isn’t fully clear to me), and published authors receive the reputational benefits of having their research franchised to a known brand.
Brand-recognition is much quicker and easier than judging things for yourself. If I buy climbing equipment, I don’t know how to test it or make good judgements about its quality. So I just look for brands that I know, trusting that they wouldn’t endanger their reputation by making an unreliable product. Similarly, hiring and promotions panels can use brand recognition when judging a large number of candidates, especially when they aren’t experts in the relevant area.
But what, in this case, does the branding tell us? I know that Black Diamond gear is likely to be pretty good, and I know pretty well what I mean by “good”. Do I know that a scholar with a publication in a leading journal does good philosophy? Do I even know what the leading journal is for a given area? I sometimes hear academics making confident pronouncements on this, but the source of their confidence is the profound unfalsifiability of their claims. If Black Diamond made bad climbing gear, we’d hear from people with broken ankles. How exactly will we hear about if it the Journal of Social Philosophy starts publishing “bad” social philosophy? Don’t get me wrong, there are plenty of academics going around saying that this or that journal is publishing bad stuff now. But if you think you can pick up a signal through that noise, it might be because you’re part of the noise.
Even if we could agree on what makes for “good” philosophy, it’s a hard road to argue that peer review selects for it. It’s a common joke, because perfectly true, that none of the great works of philosophy would have passed peer review. Plato’s dialogues? No bibliography, poor argument-structure. Ibn Sina? Apologies, Dr. Sina, but we can’t accept detailed commentaries on already-published material. Zhuangzi? No joke-publications please. Speaking of Zhuangzi, here is a fascinating book by Ellen Marie Chen: scholarly, original, creative, engaging, cross-cultural, and in many other ways unpublishable by any academic press. Even famous articles in philosophy–those by Gödel and Turing, which philosophers always bring up when trying to justify their discipline’s existence to the public–were published before the days of blind peer review.
It is well known in the sciences that peer-review filters quality very poorly, if at all. This blog post describes some discouraging findings in the natural sciences:
Scientists have run studies where they deliberately add errors to papers, send them out to reviewers, and simply count how many errors the reviewers catch. Reviewers are pretty awful at this. In this study reviewers caught 30% of the major flaws, in this study they caught 25%, and in this study they caught 29%. These were critical issues, like “the paper claims to be a randomized controlled trial but it isn’t” and “when you look at the graphs, it’s pretty clear there’s no effect” and “the authors draw conclusions that are totally unsupported by the data.” Reviewers mostly didn’t notice.
The author, Adam Mastroianni, also points out: “That debunked theory about vaccines causing autism comes from a peer-reviewed paper in one of the most prestigious journals in the world, and it stayed there for twelve years before it was retracted”. If peer review works this badly in the natural sciences, where disciplinary standards are a lot clearer, I can’t see how it could be effective at identifying the mellifluous quality of “good philosophy”.
One person told me that peer review in logic and philosophy of mathematics serves to check if proofs are valid. Really? Check out the mathematical example on p.340 of my article here (if you can access it; if you can’t, find a free version here, though in that one I was able to correct the mistake—ask again what we need journals for). It would have been nice for me if peer review had picked that up before it was indelibly printed with my name on it. But no reviewer mentioned it. Instead I had many suggestions to add new sections to my article discussing topics of no relevance to my argument but clearly of great interest to the reviewers. I was also encouraged to cite some more papers–not exactly related to my discussion, but surely I could change my discussion a bit to make it related–we all need citations, don’t we? 😉
If peer review doesn’t select for “good” philosophy and doesn’t filter out obvious errors then what does it select for? One answer is: conventionality. Here, too, I can’t do better than quote Mastroianni:
you used to be able to write a scientific paper with style. Now, in order to please reviewers, you have to write it like a legal contract. Papers used to begin like, “Help! A mysterious number is persecuting me,” and now they begin like, “Humans have been said, at various times and places, to exist, and even to have several qualities, or dimensions, or things that are true about them, but of course this needs further study (Smergdorf & Blugensnout, 1978; Stikkiwikket, 2002; von Fraud et al., 2018b)”.
And peer review doesn’t just filter out interesting, individual, or original writing: it stops people from producing it in the first place. Mastroianni again:
[…] just knowing that your ideas won’t count for anything unless peer reviewers like them makes you worse at thinking. It’s like being a teenager again: before you do anything, you ask yourself, “BUT WILL PEOPLE THINK I’M COOL?” When getting and keeping a job depends on producing popular ideas, you can get very good at thought-policing yourself into never entertaining anything weird or unpopular at all. That means we end up with fewer revolutionary ideas, and unless you think everything’s pretty much perfect right now, we need revolutionary ideas real bad.
I couldn’t agree more that we need revolutionary ideas. But in philosophy I think it’s even more extreme. Real philosophical ideas are inherently revolutionary. The whole point of philosophy is to ask questions that go deeper than other disciplines, to question what others don’t, to work at the limits of human understanding.
Peer review in philosophy, on the other hand, is as counterrevolutionary as it gets. Academics in general tend to have egos that are both big and fragile (it comes, I think, from doing very well at school but ending up only in the top 20% of earners). Academic philosophers, lacking much of the social status that accrues to other disciplines, are even more bitter and fragile. The last thing they will want to admit is that some stranger has thought of something important that they hadn’t thought of, or undermined a conventional opinion that they hadn’t questioned, or figured out a better way to address a problem than they had conceived. This isn’t conscious; I think it’s an unconscious bias that develops to protect our egos. We see the world in the way our self-esteem demands. And so, when we review, we are subtly but strongly biased against anything particularly revolutionary. Admitting the value of the revolutionary is, after all, admitting the deficiency of the status quo. And if you’re selected as a reviewer, it’s more than likely because you helped to build the status quo.
Peer review thus functions on the whole, as far as I can see, to suppress originality, to enforce groupthink, to amplify bias against anything unconventional, to entrench paradigms. Of course there are exceptions, but I’m talking about the median. Worst of all, peer review filters out all personal idiosyncrasies and individual style from philosophical thought and writing.
Here is an analogy. Perhaps you’ve been to the comedy show, Banana Caberet, in South London. The show is in two parts: first, some relatively unknown comedians; second, some big names. When I went, the first half was full of original, interesting acts: neurodivergent, culturally diverse comedians doing music-based comedy, silent comedy, conceptual comedy, etc. The second half was three men who told jokes about how their wives didn’t put out anymore. Three men who told what we might call The Joke, as Aristotle was once The Philosopher. The acts in the first half got genuine laughs from subsections of the audience. But they weren’t for everyone–that’s what made them interesting. The men with The Joke got a tired chuckle from the majority of the audience. Ah yes, that one. I know that one. Her indoors. That’s an ok one. Ha. Now imagine if each comedian were going to be assessed by one or two members of the audience, and this was going to determine their career. Not knowing who those members would be, the optimal strategy would be to do The Joke. Aim for the median. Risk nothing.
This is how peer review appears to me to filter philosophical ideas. Anything daring or original is filtered out, either through preemptive self-censorship or in the process itself. What is left is a bland broth of featureless, technocratic, box-ticking: the philosophical equivalent of The Joke. Every possible criticism that anyone could make has to be addressed and any literature such criticism might draw upon cited. Nobody talks, thinks, or writes like this naturally. So academic philosophy has become entirely divorced from the natural process of human thought. But we are preserving a system that makes your career as a philosopher contingent on your capacity to produce this stuff, and lots of it.
Why? I think it’s those egos again. Many of us academic philosophers running the system got to where we are by succeeding under peer review. To admit that peer review doesn’t select for the best philosophy–that arguably it selects for rather bad philosophy–is to take away our bragging rights. What if your H-index doesn’t measure your intellectual proximity to Plato? What if it only measures your skill at predicting and hacking the prejudices of the median reviewer? What if all we did was tell The Joke? What happens then to our status as intellectuals? By God, that’s all we have!
I understand the fear all too well. I feel the status anxiety too. But let’s not make ourselves into a meme: Philosophers would literally rather funnel taxpayer money intended to support research directly into the pockets of publishing shareholders while doing millions of hours of unpaid labour per year than go to therapy!
I literally would not rather. Let’s kill this thing. We can spend the money we save on therapy if we need. We can learn to love ourselves even if peer review doesn’t measure anything worth measuring. And then we can think of ways to make hiring, promotion, and funding decisions on a more meaningful basis.
I have ideas about that too, but they’ll have to wait for now.
Follow up post here.
Alexander Douglas 