Sokal 2.0 as Ideological Turing Test

Normally, I avoid blogging on anything topical; I want to write for the ages, not our daily hysteria. But I’m going to make an exception for… “Sokal 2.0.” The Chronicle of Higher Education provides an overview:

Three scholars—Helen Pluckrose, a self-described “exile from the humanities” who studies medieval religious writings about women; James A. Lindsay, an author and mathematician; and Peter Boghossian, an assistant professor of philosophy at Portland State University—spent 10 months writing 20 hoax papers that illustrate and parody what they call “grievance studies,” and submitted them to “the best journals in the relevant fields.” Of the 20, seven papers were accepted, four were published online, and three were in process when the authors “had to take the project public prematurely and thus stop the study, before it could be properly concluded.” A skeptical Wall Street Journal editorial writer, Jillian Kay Melchior, began raising questions about some of the papers over the summer.

Beyond the acceptances, the authors said, they also received four requests to peer-review other papers “as a result of our own exemplary scholarship.” And one paper—about canine rape culture in dog parks in Portland, Ore.—“gained special recognition for excellence from its journal, Gender, Place, and Culture … as one of 12 leading pieces in feminist geography as a part of the journal’s 25th anniversary celebration.”

The scholars behind the hoax describe their master plan here:

Our paper-writing methodology always followed a specific pattern: it started with an idea that spoke to our epistemological or ethical concerns with the field and then sought to bend the existing scholarship to support it. The goal was always to use what the existing literature offered to get some little bit of lunacy or depravity to be acceptable at the highest levels of intellectual respectability within the field. Therefore, each paper began with something absurd or deeply unethical (or both) that we wanted to forward or conclude. We then made the existing peer-reviewed literature do our bidding in the attempt to get published in the academic canon.

Examples, in the authors’ own words:

Sometimes we just thought a nutty or inhumane idea up and ran with it. What if we write a paper saying we should train men like we do dogs—to prevent rape culture? Hence came the “Dog Park” paper. What if we write a paper claiming that when a guy privately masturbates while thinking about a woman (without her consent—in fact, without her ever finding out about it) that he’s committing sexual violence against her? That gave us the “Masturbation” paper. What if we argue that the reason superintelligent AI is potentially dangerous is because it is being programmed to be masculinist and imperialist using Mary Shelley’s Frankenstein and Lacanian psychoanalysis? That’s our “Feminist AI” paper. What if we argued that “a fat body is a legitimately built body” as a foundation for introducing a category for fat bodybuilding into the sport of professional bodybuilding? You can read how that went in Fat Studies.

Needless to say, active practitioners of “grievance studies” were displeased. But quite a few more traditional academics have been equally quick to dismiss this project’s importance. Thus, my old friend Jacob Levy remarks,

I am so utterly unimpressed by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it.

Well, unlike Jacob, I am impressed. Deeply impressed. Why? Because, on reflection, Sokal 2.0 amounts to an Ideological Turing Test. As I originally explained in 2011:

Mill states it well: “He who knows only his own side of the case knows little of that.” If someone can correctly explain a position but continue to disagree with it, that position is less likely to be correct. And if ability to correctly explain a position leads almost automatically to agreement with it, that position is more likely to be correct. (See free trade). It’s not a perfect criterion, of course, especially for highly idiosyncratic views. But the ability to pass ideological Turing tests—to state opposing views as clearly and persuasively as their proponents—is a genuine symptom of objectivity and wisdom.

My idea has inspired multiple actual tests. But frankly, none of them are in the same league as Sokal 2.0. Three scholars who held a vast academic genre in low regard nevertheless managed to master the genre’s content and style expertly enough to swiftly publish enough articles to earn tenure! Frankly, if that doesn’t impress you, I don’t know what would.

The main question in my mind: Does Sokal 2.0 primarily show that the authors are intellectually strong… or that “grievance studies” is intellectually weak? Both can be partly true, of course. But the harder the authors had to toil to achieve their goal, the less they impugn the honor of their target. So how hard did they toil? The authors’ self-account:

[W]e spent 10 months writing the papers, averaging one new paper roughly every thirteen days… As for our performance, 80% of our papers overall went to full peer review, which keeps with the standard 10-20% of papers that are “desk rejected” without review at major journals across the field. We improved this ratio from 0% at first to 94.4% after a few months of experimenting with much more hoaxish papers.

In other words, they barely broke a sweat. While you could accuse the authors of self-deprecation, this is a rare human failing. When we succeed, most of us like to highlight our own awesomeness, not the ease of our goals. While most people would have been less successful than the hoaxers, what they did was far from superhuman. And that, in turn, amply supports their main theses: the fields they hoaxed have low intellectual standards and don’t deserve to be taken seriously.

Does this mean that the subjects of race, gender, sexual orientation, body image, and so on don’t deserve to be taken seriously? Not at all. You shouldn’t blame subjects just because the fields that study them fall short. Identity is too important to be left to people who embrace their own identity. Still, until the researchers who study these subjects calm down, speak clearly, and treat dissent with civility, they will continue to produce little knowledge.

P.S. My main caveat about my positive evaluation of Sokal 2.0: I’ve seen too many hoax movies not to wonder if there’s a hoax within a hoax. Probably not, though.

This article is reprinted with permission from the Library of Economics and Liberty.