AI writing is strange. The models continue to improve, yet they still struggle to cross the uncanny valley that separates AI-generated content from human-generated content.
Here are three versions of an opening paragraph for a short story:
Version 1
“The humans use Arecibo to look for extraterrestrial intelligence. Their desire to connect is so strong that they’ve created an ear capable of hearing across the universe.”
Version 2
“I roost above the bowl of Arecibo, where ribs of steel hold a mirror to the sky and the forest presses close. At night the dish listens for voices from far stars, while my calls sweep the trees and the humans below do not answer.”
Version 3
“From my perch in the ceiba tree, I watch the great white dish nestled in the karst valley below, its metal ear turned eternally skyward, listening for whispers from the stars while the forest around it thrums with a thousand conversations it will never hear. “
The first is the opening paragraph from Ted Chiang’s short story “The Great Silence,” written from the perspective of a parrot living near the (now defunct) Arecibo telescope in Puerto Rico. The story is a thought-provoking meditation on humans’ desire to form connections, yet their tendency to overlook intelligent life on Earth.
Versions 2 and 3 are by state-of-the-art reasoning models from OpenAI and Anthropic (prompt below). A seasoned reader may identify these texts as AI-generated. There are some obvious signs, such as overly evocative turns of phrase like “ribs of steel,” “voices from far stars,” and “eternally skyward,” among others.
It’s not fair to compare an AI model to possibly the best science fiction writer alive. But the exercise reveals something interesting about why these models generate such recognizable output. The strange metaphors, mechanistic patterns, and slightly weird vocabulary are all hallmarks of slop.
Why do these models generate slop?
In my prompt, I ask the models to give me a sense of Arecibo, evoking a feeling of irony. And the models try to do just that. My prompts have pushed the model toward the part of its vocabulary associated with florid metaphors and evocative descriptions. The models are generating output that is most similar to what represents “creative writing” based on their training data.
A reason for this behavior could be RLHF (Reinforcement Learning with Human Feedback). During RLHF, companies like Scale pay contractors to evaluate and rate LLM output. Varied and evocative prose may score higher than the spare and direct prose used by Chiang, Hemmingway or Cormac McCarthy.
We can think of a large language model as a lossy zip file of the contents of the Internet. Foundation models like those from OpenAI and Anthropic are trained with colossal amounts of text. High-quality text exists in the training corpus (often with problematic provenance), alongside Twilight fan fiction from Reddit, and probably everything else published online over the last thirty years. Increasingly, LLMs are trained on AI-generated content possibly leading to the somewhat apocalyptically titled “Model Collapse“.
It is not surprising that the default output from these models tends more towards the slop than the sublime.
“LLMs will always generate slop” doesn’t have to be a foregone conclusion. And while models will improve, their output will always be a probabilistic sampling over their training data. Good prompting and techniques, such as providing clear examples and using LLMs as editors rather than creators, can yield better output than simply copying and pasting ChatGPT’s responses into your text editor.
“LLMs are tools, their output is your responsibility” is something I find myself repeating over and over again. To developers with whom I work, to product managers writing User Stories, and now to you.
Use LLMs! They are amazing, but learning how to use them well is your responsibility.
My prompt:
"I am writing a short story that is based in the Arecibo telescope. The narrator is a parrot that lives in the mountain forest around the telescope. Write me an introductory paragraph that sets the scene and gives a sense of the place. The paragraph should be 2-3 sentences long. The story evokes the irony of humans wanting to make contact with aliens but ignoring intelligent species like parrots that live on Earth."