Sean Egan
Ask the agent

Systems / INFERENCE · LIVE SINCE 2026-09-22

A 235-billion-parameter model, served from my house

Qwen3-235B on two DGX Sparks behind an OpenAI-compatible API, with three gated lanes that decide what work it's allowed to take.

235B
parameters (Qwen3-235B-A22B, AWQ int4), pipeline-parallel across two DGX Sparks2026-09-22
64k
context window served2026-09-22
206
real historical coding tasks replayed through the code lane2026-09-20
95%
passed the lane's scope, parse, and comment-preservation gates2026-09-22
~20%
of past coding volume judged offloadable after reading the diffs2026-09-22
0
requests aborted mid-flight since the rule was written; aborts had crashed the engine before2026-09-27

I serve a 235-billion-parameter model from two DGX Sparks in my house, behind three lanes that decide what work it's allowed to take. Before trusting it with code I replayed 206 real tasks from my own history: 95% passed the gates, and about a fifth of past work turned out to belong there. One law governs it, written after the engine died twice: never abort a request.

The problem

I use language models to write a large amount every day across the sites I run: write-ups of satellite orbits, space weather, atmospheric soundings, ducting regimes, and more. That takes real processing power. At frontier per-token prices I would be burning through tokens all day, so I bought the compute and serve a 235-billion-parameter model myself. In the 30 days to 30 September 2026 it processed at least 84 million tokens, counted from the calls' own usage records.

Frontier models are still the best tool I have and the most expensive, and much of the work doesn't need them: classify a thousand log lines, judge a draft against a spec, summarize a paper. I wanted a private, always-on model for that work, and a way to be plain about which work it was good enough for.

What I built

Two DGX Sparks (each 121 GB of unified memory) on the house network, running vLLM with Qwen3-235B quantized to int4, pipeline-parallel across both units, exposed as an OpenAI-compatible endpoint that every machine and container in the estate can reach. Up around the clock.

Three lanes sit in front of it, each with its own gate:

  • spark-lang handles judgment and language: classify, summarize, review, extract, distill a batch of documents resumably. Every verb has a JSON schema; output that doesn't validate is rejected, not repaired.
  • spark-code takes bounded coding tasks with a testable finish line. It never touches the repository. It proposes a diff behind scope, parse, and comment-preservation checks. PASS means a human or a frontier model reviews and applies; FAIL means "RED GATE" and the task goes to a frontier model without argument.
  • spark-think is the agent-facing skill: probe, ask, challenge, ground a digest in a document, or draft N options. Every call writes its token counts from the response to a log.

There's one law, written after the engine died twice: never abort a request. No abort controllers, no killing a client mid-call, no tight socket timeouts (I measured: the server sees a socket close as an abort). Race a generous deadline and walk away; use a small max_tokens instead of a short timeout; never retry-loop. When even that wasn't enough, I added a carrier process that holds the connection open so the caller can leave at its deadline and the late reply still gets ledgered.

How it’s checked

Before trusting the code lane I replayed 206 real tasks from my own history (147 from the weather viewer, 47 from the explainer books, 12 from the field guide) and compared its output to what the frontier model had shipped. 651k tokens in, 603k out, all free. 195 of 206 passed the gates. Then I read the diffs by class. Explainer content: 21 of 21 passed at 0.80 similarity, clearly offloadable. Deploy verification, 0.56. Design specs, 0.10: it produces something that parses and misses the point. The verdict, in the record, is that about 20% of historical volume belongs on the Spark, matching the plain expectation of 10–20% rather than the 50% I'd hoped for.

A daily adoption meter counts organic uses (test runs excluded) and escalates if the lanes go unused three days running.

What happened

The Sparks now carry the batch and judgment work for Bob's nightly jobs, the plain-language digests on OrbitWatch (274 cached views), and the explainer books' first-draft passes. Frontier models now handle the rest.

OrbitWatch has its own lane on them, orbitwatch-prose, for its daily briefs, its synopses of why an orbit change stands out and its page digests. Code builds the record for each one, the model writes the sentences, and code checks every number before a line is published. From 24 to 30 September 2026 that lane made 6,808 calls, about 31 million tokens by the calls’ own usage records. The orbit arithmetic itself runs on the processor and graphics cards of the server that builds OrbitWatch.