What I keep seeing
- The model thinks in language, so it presents in language.
- When the work gets hard, it stops writing for the reader.
I think these are one problem seen twice. Nothing in the process that produces a draft stands for the person who will take it in. In the first case the model doesn't see the page the way a person does. In the second it stops seeing the reader at all.
How it presents information
It reaches for the medium it thinks in.
What a first draft looks like
When I ask a model to build a website or draft a research paper, the first draft captures the ideas I want to convey, and it's well organized. That part is good, and it's much of why I work this way.
The trouble is the medium. With little data to show, the model pays attention to layout and graphics. Once there's real data, or real ideas to convey, the layout holds up and the effort goes into generating text. A website comes back as pages of text with a few graphics scattered through. A research paper comes back with no figures, graphs or diagrams at all.
My read is that these models think and reason in human language. When they're asked to convey information, they reach for the medium they think in.
People take in data visually first and through language second, and the model presents it the other way around.
I know multimodal models exist, and the models I use can read a chart and write the code that draws one. What I'm describing is what a model does by default when it's asked to present information. I've seen it across many builds this year.
First drafts
Two research papers on satellite orbits, about 5,800 and 7,700 words, with no figures in either.
After my style rules
The same two papers at about the same length, with 6 figures in one and 8 in the other.
Word and figure counts
In the same week, the first drafts of twelve one-page proposals for a Navy AI challenge ran 570 to 720 words each, with no figures or tables. The technical review that went with them ran about 3,000 words, and a combined review book about 9,250, with no figures in either. A memo to a commanding officer ran about 3,000 words over 11 pages with no figures, and its section titles named the method where the reader needed the problem.
My style rules now say to plan and render the figures before writing the results. With that rule in place, a companion research memo went from 7 figures to 18. The standard my style guide points to is my own 2020 paper, which has ten figures and four tables in about 8,000 words.
Two objects passing in orbit
OrbitWatch is the orbit-analysis system I built on the public satellite catalog. One of the things it does is work out how close two objects in orbit came to each other. The first answer I got was a paragraph.
What I first got
Written for this page in the style of that first answer. The numbers are made up, and they match the illustration below.
The two objects reached their minimum separation of 2.0 km at 14:32:07 UTC. At that moment their relative speed was 8.3 km/s. The separation fell from about 500 km one minute before closest approach and grew at the same rate afterward, so the pass was brief and nearly symmetric. The estimate depends on the age of the element sets, and the true minimum could differ by several kilometers.
When I asked for a diagram, I got one. It didn't occur to the model that watching the two objects approach would help a person understand what the numbers were saying.
Then I told it that the page had to help a person see how the objects moved in time. That brought a slider that moves the two objects along their paths and a graph of the distance between them over time. The page also showed how fast that distance was changing, and the objects' momentum and kinetic energy.
That was what the data was saying. The paragraph had the numbers and none of the motion.
Illustration
60 s before closest approach 498 km apart
This is how most visuals go. I steer the model toward a visual, and then I steer how it builds one, because it isn't thinking that way on its own. The first attempt is often basic, or a little strange.
What I have to tell it every time
To a model, a wall of text is easy. To a person it's distracting and hard to take in. My standing instruction to every agent I run is "no walls of text," and I still have to repeat it on every job. The labs' own guidance says the same thing: put the format in the prompt, and say it again later, because the rules fade over a long session.
The model also doesn't know that information can be layered. A small visual can stand for a piece of information, and a person who wants more clicks into it for a tooltip, a popover or a fold. Unless I direct otherwise, everything it builds is flat. It wants to hand the reader all the data at once, where I would put some of it one click down.
Time works the same way. The model understands time as a concept, and it doesn't carry that into how it presents things. People like to see how things change, and the model doesn't think about change over time the way we do.
When I read my style guides back, they're a list of reminders about the reader. They tell the model to plan the figures before writing the results and to keep one line in view with the rest behind a fold. They also say to leave no single word alone on a line, and to show how things change over time. I wrote those rules for the model. A person designing a page for another person follows most of them without being told.
I think I have to say these things every time because nothing in training says them. I can't see inside training, but nothing in how these models behave suggests it rewards how a person takes in information. In particular, we approach data visually before we approach it through language.
I think the training objective is missing a term, one that accounts for how a person will interpret what the model has presented.
There's a sweet spot where a model's output combines text and visuals into something sophisticated that a person takes in easily, and nothing in training pulls toward it yet. I have thoughts on how to get there, and I'd like to work on it.
Who it writes for
When the work gets hard, the reader drops out.
It starts writing for itself
The second habit shows up on deep technical work in science and engineering. The model's language changes. It fills with jargon and coined terms, abstract nouns stack up, and paragraphs close on aphorisms. The grammar and the word choice get strange and hard to follow.
The ideas are all there. It reads as if the model is presenting the data to itself, thinking out loud, rather than addressing an audience. Much of what my prose lint catches across my sites is that process leaking into the finished page.
I can follow it because I've lived inside the project for months. Handed to someone else, even an expert in the field, it would be hard going.
I see it with each of the frontier models I use (Grok, GPT and Claude) and with open models too, especially large ones. In my experience it comes easily past a couple of hundred billion parameters.
What it looked like
These are sentences from the first drafts of my two orbit papers. I asked for those drafts, and the ideas in them were sound.
"The gate closes in practice, which is the only evidence that it is real."
"The station-keeping odometer reads far too low by construction."
"This is the prior-art review's second design requirement discharged with data rather than argument, and it is the clearest positive contribution this paper makes."
"Gate" and "odometer" are words the project coined for its own use, written as if they were standard terms. The third sentence is about the paper's own contribution where the reader wanted the result.
After the style rules, "registered" and "registration" went from 71 uses to 4 in one paper and from 113 to 4 in the other. Em dashes went from 91 to 1 and from 136 to none. The slogans are gone, and the papers are still long and dense.
The first drafts for a Navy AI challenge read the same way. They were dense with acronyms and defensive, and they spent more words on what the proposal did not claim than on what a user would get.
I wrote up how the papers were revised in a field note, The paper that sounded like a model talking to itself.
When agents talk to agents
I suspect the driver is parallel agents. On a big project my lead agent hands work to subagents and reads what they send back. The more of that traffic there is, the more jargon-heavy and difficult the result becomes. It's as if agents talking to each other push the style along.
I can't prove the cause, but I have a guess. In a multi-agent build, most of the text a model reads was written by another model, for a model. Shared context lets a coined term stand in for a paragraph, the way a lab develops its own shorthand.
People do this too. A research group talks in shorthand all day, and when someone from outside walks in, the people in the room switch register without thinking about it.
Nothing in the loop tells the model that the outsider has walked in.
The one thing I've found that works on both problems at once is letting the agent see what I see. When a page or a paper has everything we need and still hurts to look at, I send a screenshot and ask what's wrong with it. The agent points out the same things I would have, and because it then briefs its subagents in its own words, the fix comes out better than when I list the faults myself. My read is that the screenshot makes it look at the page as a person does, and a brief in its own words survives the trip to other agents better than one it was handed.
This one needs solving too, and I'd like to be part of that work.
Both habits come from the same place, a reader the model can't see. Noticing that changed how I direct agents, and it's why my style guides exist. It's also what I'd most like to work on next, with the people who train these models. If you're working on this, or you have thoughts about it, I'd like to hear from you.
sean@theinformed.org