These new “artificial intelligence” programs don’t know what they’re talking about

I’m sure you’ve seen things like ChatGPT in the news: programs that can carry out pretty convincing conversations. They are known as Large Language Models (LLMs) and are frequently referred to as being Artificial Intelligence (AI) — but I really don’t like that designation as it implies some understanding.

To try this out, I asked OpenAI Chat this question: “What are the contenders for the title of Biggest Dinosaur, and what is the basis for their various claims?” I picked that because I know a bit about the answer, and in fact wrote an answer myself back in 2015.

Here’s what it came up with:

There are several contenders for the title of “biggest dinosaur,” and their claims are based on different pieces of evidence. The most commonly cited contenders are:

1. Argentinosaurus: This is considered to be one of the largest known dinosaurs, with estimates of its length ranging from 110-130 feet (33-40 meters) and a weight of around 100-110 tons. Its size is based on a few fragmented bones, including vertebrae and limb bones, which were found in Argentina.
2. Brachiosaurus: This dinosaur is estimated to have been around 85 feet (26 meters) long and weighed around 80-100 tons. Its size is based on several well-preserved specimens that have been found in North America, including a nearly complete skeleton.
3. Puertasaurus: This dinosaur is estimated to have been around 85-100 feet (26-30 meters) long and weighed around 80-100 tons. Its size is based on a few bones, including a femur, that were found in Argentina.
4. Supersaurus: This dinosaur is estimated to have been around 110-125 feet (33-38 meters) long and weighed around 80-100 tons. Its size is based on a single specimen, a set of vertebrae, that was found in the US.
5. Diplodocus: This dinosaur is estimated to have been around 90-115 feet (27-35 meters) long and weighed around 20-40 tons. Its size is based on several well-preserved specimens that have been found in North America.

It’s worth noting that the size of these dinosaurs is approximate and there’s still debate among paleontologists. Also, the weight of these dinosaurs is based on estimations, not real measurements.

At first glance this is a decent answer. It’s certainly better than just saying “Argentinosaurus weighed 100 tons”. But if you know your sauropods, you’ll immediately recognise that this answer is riddled with inaccuracies:

• The Argentinosaurus entry is actually pretty good, although I’ve never seen a 110 ton mass estimate and modern estimates tend to be rather lower than the specified range.
• The Brachiosaurus mass estimate is way off, two or three times as high as it should be.
• More importantly, no remotely complete Brachiosaurus specimen has ever been described, and size estimates are based primarily on its Tanzanian relative Giraffatitan.
• There is no known femur of Puertasaurus. (When I corrected the AI on this, it told me instead that there is a pubis known.)
• Supersaurus is not based on a single specimen, and both of the main specimens that have been described contain plenty of appendicular material.
• The Diplodocus length estimate is a bit inflated, but otherwise not bad. But it’s not clear what it’s doing in a list of five biggest dinosaurs.
• The answer omits some very strong contenders, including Dreadnoughtus and Patagotitan.
• It doesn’t really address the second part of my question — e.g. Supersaurus has a good claim to be longer, but not heaviest; the converse is likely true for Argentinosaurus.

Now here is the real problem: the LLM does well enough to fool people. If it was nonsense from start to end, there would be nothing to fear here, but the plausibility of the answers and the authoritative tone in which they are given lends the many mistakes a credibility that they do not deserve.

Having seen this sort-of-convincing-but-very-wrong reply in a field that I know something about, I would be very very cautious about trusting an LLM to teach me about a field I don’t already know. I’m guessing its replies about space flight, quantum physics and Medieval French literature are going to be similarly flawed (but also, worryingly, similarly convincing to those such as myself who don’t know better.)

There is a very fundamental reason for all these mistakes: as I implied above, LLMs do not understand anything. They just know what phrases occur close to other phrases. They can do amazing things with that one trick, and I can see them being useful as discovery tools. But we’ll go badly wrong when we start trusting them as anything more than a bright but ignorant kid offering suggestions.

So for all the talk of AI having taken huge leaps forward in the last couple of years, I don’t think any such thing has happened. We’ve just got much better at generating plausible text. But there’s no advance in actual understanding.