Field note · 2026-08-27
The tool catalog that was never sent
I was told tool definitions were costing 65,811 characters on every call. They weren't, and chasing the real cost taught me to print what is sent next to what is recorded.
- 4
- tools sent to the model, of 57 in the catalog2026-08-27
- 1,700
- characters of tool definitions per call, median2026-08-27
- 11.2% → 0.03
- discovery calls, as a share of tool calls, then per tool use2026-08-27 → 09-24
- 16
- characters by which one task pack missed its cap and was dropped whole, for a month2026-09-10
What I saw
Bob was slow, and a cost report said his tool catalog weighed 65,811 characters per call. The obvious fix was to cut tools.
What caused it
Nothing, because the number was wrong. Bob's tools are deferred: the gateway knows all 57, and the model is sent four meta-tools to search, describe, and call the rest. Across 449 compiled contexts in a week, 418 shipped those four tools and 1,700 characters. The 65,811 figure was the full catalog as recorded for observability. 97.4% of it never left the box.
The real cost was round trips. In 201 sessions and 5,072 tool invocations, 569 were the model looking up a tool before using it. That is 11.2% of all tool calls spent on discovery.
What I changed
I pre-rendered full signatures for the nine tools that accounted for 76% of lookups, so the model could call them without asking. One tool had been listed as primed and never rendered, because its signature was 683 characters against a 400-character cap; nothing had complained. A month later discovery was down to 0.03 calls per tool use.
I kept the tool list pinned and byte-stable on purpose, because that is what keeps the prompt cache warm. Turning tool search off would have added about 55,000 characters per turn to save almost nothing.
The same whole-or-nothing pattern turned up elsewhere. A task pack had been dropped in full since August for being 16 characters over its cap. The skill catalog had silently degraded to names only, so the model was choosing skills by filename.
What I'd tell another team
Every cost report should print what was sent beside what was recorded, and a test should fail if the two are ever swapped. And look for caps that drop the whole item, because they leave no trace.