The Case of the AI-Generated Giant Rat Penis

An illustration featuring a rat with a cross section of a giant penis set off a firestorm of criticism about the use of generative artificial intelligence based on large language models (LLMs) in scientific publishing. The bizarre illustration was embellished with nonsense labels, including one fortuitously designated as “dck.” The article on rat testes stem cells had undergone peer review and editorial vetting before being published in February by Frontiers in Cell and Developmental Biology.

“Mayday,” blared longtime AI researcher and critic Gary Marcus on X (formerly known as Twitter). Vexed by AI’s ability to abet the “exponential enshittification of science,” he added, “the sudden pollution of science with LLM-generated content, known to yield plausible-sounding but sometimes difficult to detect errors (‘hallucinations‘) is serious, and its impact will be lasting.”

A February 2024 article in Nature asked, “Is ChatGPT making scientists hyper-productive?” Well, maybe, but Imperial College London computer scientist and academic integrity expert Thomas Lancaster cautioned that some researchers laboring under “publish or perish” will surreptitiously use AI tools to churn out low-value research.

A 2023 Nature survey of 1,600 scientists found that almost 30 percent had used generative AI tools to write manuscripts. A majority cited advantages to using AI tools that included faster ways to process data and do computations, and in general saving scientists’ time and money. More than 30 percent thought AI will help generate new hypotheses and make new discoveries. On the other hand, a majority worried that AI tools will lead to greater reliance on pattern recognition without causal understanding, entrench bias in data, make fraud easier, and lead to irreproducible research.

A September 2023 editorial in Nature warned, “The coming deluge of AI-powered information must not be allowed to fuel the flood of untrustworthy science.” The editorial added, “If we lose trust in primary scientific literature, we have lost the basis of humanity’s corpus of common shared knowledge.”

Nevertheless, I suspect AI-generated articles are proliferating. Some can be easily identified through their sloppy and flagrant unacknowledged use of LLMs. A recent article on liver surgery contained the telltale phrase: “I’m very sorry, but I don’t have access to real-time information or patient-specific data, as I am an AI language model.” Another, on lithium battery technology, opens with the standard helpful AI locution: “Certainly, here is a possible introduction for your topic.” And one more, on European blended-fuel policies, includes “as of my knowledge cutoff in 2021.” More canny users will scrub such AI traces before submitting their manuscripts.

Then there are the “tortured phrases” that strongly suggest a paper has been substantially written using LLMs. A recent conference paper on statistical methods for detecting hate speech on social media produced several, including “Head Component Analysis” rather than “Principal Component Analysis,” “gullible Bayes” instead of “naive Bayes,” and “irregular backwoods” in place of “random forest.”

Researchers and scientific publishers fully recognize they must accommodate the generative AI tools that are rapidly being integrated into scientific research and academic writing. A recent article in The BMJ reported that 87 out of 100 of the top scientific journals are now providing guidelines to authors for the use of generative AI. For example, Nature and Science require that authors explicitly acknowledge and explain the use of generative AI in their research and articles. Both forbid peer reviewers from using AI to evaluate manuscripts. In addition, writers can’t cite AI as an author, and both journals generally do not permit images generated by AI—so no rat penis illustrations.

Meanwhile, owing to concerns raised about its AI-generated illustrations, the rat penis article has been retracted on the grounds that the “article does not meet the standards of editorial and scientific rigor for Frontiers in Cell and Developmental Biology.”

The post The Case of the AI-Generated Giant Rat Penis appeared first on Reason.com.