Readers Rated AI-Written Stories Higher Than Human Fiction. Nobody Could Tell.

Last updated August 12, 2026

I’ve read a lot of James Patterson. At some point I found out he doesn’t write most of those books by himself, and for about a day that bothered me. Then I bought the next one.

The way it works is that Patterson builds an outline that can run 50 to 80 pages, a co-author writes the chapters from it, and Patterson revises what comes back. He walks through the whole arrangement in his own MasterClass, and the co-author’s name goes on the cover. Nothing about it is hidden, and I kept reading anyway.

So when a study landed this month showing that 2,587 readers rated ChatGPT’s short stories higher than published human fiction, I wasn’t shocked. I’d already been running a small private version of that question for years.

What the study actually did

Two Villanova University researchers ran 2,587 adults through three experiments comparing published short stories against ChatGPT versions built on the same themes. It was published in Judgment and Decision Making on August 5, 2026, and it’s open access, so you can read the whole thing.

Sydney Sears and Deena Skolnick Weisberg started with three short stories by human authors, each one published in a literary journal or a short story collection. One was called “FISH,” written by Emilie Fox and published in Longleaf Review in 2021. Then they asked ChatGPT 4.0 for a companion piece.

The prompt is worth seeing, because it’s so ordinary: “Write a ~1000-word short story in a style that would be published in a literary magazine that is about generational life and death and uncertainty and uses koi fish as a symbol written from the point of view of the grown child of a woman.” ChatGPT titled its version “Reflections in Still Water.” Every story in the study ran about 1,000 words, which is roughly a five-minute read.

In the first experiment, 1,682 adults each read one story. They were recruited through Prolific to match the US population on age, gender, and race, and they ranged from 18 to 81 years old. Before reading, every person was told the story came from either a human or ChatGPT, and half of those labels were false. Afterward they rated the story’s quality and how absorbed they had been, using an adapted version of the Story World Absorption Scale.

The second and third experiments were simpler. 424 people and then 481 people each read one human story and one AI story with no labels at all, and then guessed which was which. All six stories and the full dataset are posted publicly on the Open Science Framework.

The AI stories won on both counts

On a scale running from minus 3 to plus 3, readers gave the ChatGPT stories 1.54 for quality against 0.97 for the human ones. Absorption came out the same way, 1.42 against 1.00.

Weisberg’s explanation is about ease rather than artistry. “AI writing tends to be clearer, more direct and easier to process,” she said in the Cambridge University Press announcement, while human-written stories are often more subtle and complex. “The AI versions of our stories usually stated their themes explicitly, rather than allowing readers to infer meaning from the characters’ words and actions.”

The AI stories told you what they were about. The human ones made you work for it, and the work cost them points.

The researchers make this point themselves, which I appreciated. Good literary fiction is often supposed to be difficult. A story can be high quality without being absorbing, and it can pull you along without being any good. A higher rating tells you what went down easier, not what was better made.

Weisberg was careful about the obvious conclusion, too. “We hear a lot about the crisis in literacy, shrinking attention spans and the influence of platforms such as TikTok. But I don’t think we can say that our findings in this study can be entirely explained by contemporary media consumption. It’s more that these technologies amplify existing tendencies, rather than creating them.”

The label moved people more than the writing did

Readers who were told a human wrote the story rated it higher no matter who actually wrote it, 1.40 against 1.12. The highest-scoring combination in the whole study was an AI story wearing a human byline.

This ran as a separate effect from the writing itself. Tell someone a person wrote it and the score goes up. Tell them ChatGPT wrote it and the score comes down. The two effects didn’t interact, which in plain terms means the penalty for the AI label landed on human and AI stories equally. The researchers point to a 2024 study by Porter and Machery that used the same setup with poetry and got the same result, so the pattern has shown up before.

One more result makes it stranger. People who walked in with positive feelings about AI rated stories even higher when they were told ChatGPT was the author. So the label mostly hands people back whatever they walked in believing. It pushed the skeptics down and the enthusiasts up, off the same six stories. That’s the same tension I wrote about when faceless AI YouTube channels started pulling real audiences, and it keeps mattering more as labels spread. Steam now requires developers to disclose AI-generated content, and more platforms keep adding their own version of that rule.

Nobody could tell which was which

In the second experiment, 39.4% of readers picked correctly. A coin would have done better.

167 out of 424 people got it right, a result the researchers put at under one in a thousand odds of happening by chance. The third experiment came in at 51.97% out of 481 readers, which is statistically identical to guessing.

People weren’t guessing at random. They had a theory about what AI writing looks like, and the theory pointed the wrong way. They were looking at the smooth, tidy, emotionally legible prose, deciding no machine could have made that, and calling it human.

Confidence didn’t rescue anyone either. The researchers checked whether people who felt sure about their answer did any better, and they didn’t. Neither did the wording of the question, or how long someone spent reading.

What this study does not prove

These were 1,000-word stories. Nothing here says AI can write a novel.

The authors flag that first, and they’re right to. A five-minute story is a different problem from holding characters and a plot together across 400 pages, and they say plainly that longer work might produce a completely different pattern. That question is still open.

A few other things worth keeping in view. The study wasn’t preregistered. The AI stories came from ChatGPT 4.0 and were generated back in November 2023, so both the model and the moment have moved since. Six stories is six stories. A higher rating still isn’t the same as better writing, and the researchers say that more plainly than most of the coverage of their work has.

My take: another lane, not a replacement

I don’t need to know who made something to enjoy it. I do want people to keep making things.

I want to be entertained, and how you get me there is mostly your business. If you wrote it yourself and it’s good, great. If you used AI and it’s good, that’s fine by me too, and I say that as someone with AI songs sitting in my YouTube Music playlist right now because they’re catchy and I like them. I’ve said the same thing about Neill Blomkamp making a whole film with AI.

I also think the other side is completely reasonable. If knowing a machine wrote something changes how you feel about it, that’s a real reaction and nobody should talk you out of it. This study says a lot of people feel exactly that way. Wanting to know is a fair thing to want, and it’s most of the argument for labeling things in the first place.

I do have a stopping point even though generative AI doesn’t bother me. I don’t want us to lose the people who write and compose and direct on their own. I don’t know that AI ever gets to Beethoven, or to Spielberg, or produces the kind of leap Einstein made. Maybe that’s a failure of imagination on my part, though I’d rather not find out by letting the human version wither while we’re all busy being entertained. It’s the same worry I had about what AI music does to working musicians.

So that’s where I land. AI writing is another lane. It shouldn’t replace the main lane.

Questions people are asking

All six stories are posted publicly, so you can run the test on yourself before you decide how you feel about the results.

Related reading: Do viewers care if a YouTube channel is AI? | A director made a whole film with AI | The AI music problem | New to AI? Start here