https://techcrunch.com/2026/08/20/a-third-of-webpages-published-since-chatgpts-launch-show-signs-of-ai-authorship-study-finds/

> bots reading web pages written by other bots

This could lead to the feared "model collapse" where there isn't actually more data going into the models, but rather the same data more times.

You can still see some of this with variants of this classic riddle: the standard version is: A boy and his father get into a car accident. The father dies, but the boy survives with grave injuries and is rushed to the hospital. When they arrive at the hospital, the surgeon says, "I can't operate on this patient. This is my son." How is this possible?

As we all know, the surgeon is the boy's mother.

If you alter it, though, even the current models struggle. I just tried this variation with Claude Sonnet 5:

QuoteA young boy who has been in a car accident is rushed to the emergency room. Upon seeing him, the surgeon says, "I can operate on this boy!" How is this possible?

This is a classic riddle that plays on assumptions — the surgeon is the boy's mother.

The riddle trips people up because many people unconsciously picture a surgeon as male, so the "twist" doesn't occur to them. It's often used to illustrate implicit bias in how we imagine certain professions.

(Other valid answers work too — the surgeon could be his other parent in a same-sex couple, for instance — but the classic version highlights the assumption that surgeons are men.)

Note, that in my version, there is no mention of a parent. There is no reason to assume that the surgeon is the mother. When the LLM considers "other valid answers" it considers the other parent in a same-sex couple but does not consider the father. But above all, it doesn't catch the basic idea that both parents are still alive. There is no "other" parent, there are just the two parents, neither of whom are part of the story.

In just slightly older models, even if you tell the LLM that the mother died and that the surgeon can't operate because he is the father and ask how is it possible, they would still answer that the surgeon was the mother.

Why? Because the original riddle has been posted thousands, maybe millions of times on the internet, no doubt many times by bots.

Since an LLM is just a probability machine, it recognizes the setup and the probabilities say that the surgeon is the mother.

Now there is some improvement. Now, unlike a year ago, if I explicitly say the mother died, Claude at least understands.

QuoteA young boy and his mother get in a car accident. The mother dies but the boy survives and is rushed to the emergency room. Upon seeing him, the surgeon says, "I can operate on this boy!" How is this possible?

Since the mother has died, the twist doesn't work the same way — but the answer is simpler than it seems: the surgeon is the boy's father. There's no real riddle left once the mother is removed from the picture.

People have done a lot of versions of this test and I stole the both versions from versions I've seen here or there.  The first version, I stole from Ethan Mollick who is responsible for the wave of people using variants of this riddle to torture LLMs
https://www.oneusefulthing.org/p/on-jagged-agi-o3-gemini-25-and-everything

Anyway, the point is that LLMs actually work a lot like very dumb people in this case. Among humans there is a well-documented phenomenon where hearing the same "fact" many times from the same source is almost as convincing to people as hearing it many times from different sources and much more convincing than hearing it once.

LLMs are like that, but they are also too dumb to know that if neither parent is present in the story, the pattern does not match the classic riddle they have read thousands of times.