Field note · 2026-09-11
93% of the bill was history
I assumed tool calls were flooding my agent's context. I measured two weeks of real conversations and the tool results were less than one percent.
- 31.5M
- input tokens over 14 days of real conversations2026-09-11
- 93.2%
- of that was history re-sent each round2026-09-11
- 0.7%
- was tool results2026-09-11
- 86%
- of memory lookups asked for something already in context2026-09-11
What I saw
Bob compacted in the middle of a conversation and told me there had been too many tool calls. Most conversations felt slow, with waits around a minute.
What caused it
I pulled two weeks of real traffic: 235 turns from people, in 50 sessions, with test traffic excluded. 63% of turns used no tools at all, and a median tool-free turn still cost 51,900 input tokens and 17 seconds. The 37% of turns with several model rounds made up 75% of the bill.
Of 31.5 million input tokens, 93.2% was conversation history replayed on every round. The fresh material for the turn was 6.1%. Tool results were 0.7%. The biggest single item was the context packet injected each turn, a median of 33,105 bytes, then replayed verbatim to keep the cache prefix stable. In one text-message session 76% of the bytes sent were that envelope and 2.7% were typed by people.
One alarm inside the incident was false. A notice said 343,750 tokens had come from 102 bytes of typing. It was an accumulator that never reset between turns, adding four rounds together.
What I changed
The per-turn packet went from about 46,000 characters to between 6,500 and 19,000. The tool-result clamp dropped from 32,000 characters to 6,000. A packet that was being served twice was removed, cutting first-turn input by 24%.
I also changed how tool results are clipped. The old clip had saved 0.7% of tokens and cost extra rounds, because the model went back for what was cut. Now the wrapper is reduced first and the result is delivered whole if it then fits; anything larger becomes an excerpt with a handle to fetch the rest.
What I'd tell another team
Measure before you cut, and measure real traffic with your own tests excluded. Separately, I ran a size-versus-accuracy test: from 3,000 to 46,000 tokens of context, accuracy held at 180 of 180 and speed fell by 7.5 times. The extra context cost time and added no accuracy.