Research / Overview
What I am researching, and where it stands
I research what machine learning and public data can measure, and with what error, for the work I know best: weather at sea, objects in orbit, and the language models I use to build and check both. Four lines are running, on graphics cards at home, a pair of small AI servers and a supercomputer. As of 27 October 2026, the two orbit papers are under review, the weather nowcast beats the standard open method in all five regions tested, a language-model referee catches 3 of 51 known problems in two of my papers, and the token study closed with no gain in 270 runs and 177 more that rechecked its one lead.
- 2
- orbit papers under review at Acta Astronautica, with Derek Conklin2026-09
- +0.038 to +0.074
- nowcast skill over the standard open method, in all five regions tested2026-09-24
- 3 of 51
- known problems in two papers caught by a prompted language-model referee2026-10-04
- 270
- token-study runs, with no gain over the plain lookup table2026-10-03
Four lines of research
- Building each word’s first vector from shared partsClosed, a negative result Whether a language model handles unseen combinations better when each token’s first vector is built from shared parts chosen by context. 270 runs found no gain over the plain lookup table; 0 of 177 follow-up runs on the one lead have finished. 12 Oct 2026, 20:13 UTC
- Language models that check papersRunning Training an open model to referee a paper by recomputing its claims, and a second model that cleans machine-written prose without changing a number. A prompted 235-billion-parameter model caught 3 of 51 known problems. The cleaner’s first fine-tune has been running since 09:30 UTC on 8 October 2026. 8 Oct 2026, 10:56 UTC
- Satellite orbits from the public catalogTwo papers under review What the public satellite catalog can measure with a stated error. Two papers with Derek Conklin are under review at Acta Astronautica; 16 supercomputer studies have met 36 of 95 pass lines. 8 Oct 2026, 02:12 UTC
- Weather models for nowcasting and satellite imageryRunning Radar an hour ahead, radar from satellite where none exists, and sharper satellite imagery. 90 runs at home and 19 on the supercomputer; the nowcast beats pySTEPS in five regions, and one result was retracted. 27 Oct 2026, 05:08 UTC
New lines
The token study found no gain from changing how a model reads its input, so five studies now look inside the transformer block, at the feed-forward layer that stretches, bends and squeezes each token’s numbers.
- Which activation shape does a network choose?RunningEach channel of a small language model learns its own activation shape, to see whether the shapes change with depth and whether any matches SwiGLU, the gated unit most current models use. grid being written, about 6 node-hours.
- Should context reshape the feed-forward layer?RunningThree ways of letting the rest of the sentence change the weights a token meets, tested on the token study’s unseen pairs. grid being written, about 3 node-hours.
- Can one layer multiply two features?ClosedThe feed-forward weights are read twice, the second time with each pair of channels turned 45 degrees, so that a squared activation forms products of two features. The test is whether a model can then apply a multiplier it never saw in that role. grid being written, about 8 node-hours.
- Does reading one matrix from several angles help?ClosedThe same feed-forward weights are read through four fixed random rotations and averaged, then read again through rotations they never saw in training. grid being written, about 6 node-hours.
- Do trained weights carry what the angles taught them?ClosedFeed-forward weights trained under four rotations are moved into a plain model, and the reverse, and held fixed while the rest of the model trains around them. grid being written, about 2 node-hours.
Three kinds of work
- Work meant for publication
- A paper, a note or a public data release is the end point. The two orbit papers, the nowcast comparison with pySTEPS and the token study are this kind.
- Work for our own tools and data
- The result goes into something people use: OrbitWatch, the weather viewer’s layers, and the cleaner, which is meant to check every page and paper before it goes out.
- Work that answers another project’s question
- A short study run to settle a question that came up elsewhere, such as testing the referee against the outside review of the two orbit papers.
Where it runs
- At home
- Two NVIDIA RTX 4080 graphics cards train the weather models and a third tests them.
- The Sparks
- Two NVIDIA DGX Spark units in my house serve Qwen3-235B, an open model of 235 billion parameters, to every agent I run, at no cost per token. The referee’s prompted runs used it.
- The supercomputer
- Nodes of 128 CPU cores with four NVIDIA A100 graphics processors (40 GB each), and nodes of 256 cores with four NVIDIA H100 processors (80 GB each). For fiscal year 2027 I asked for about 5.9 million core-hours across two projects, one for weather and ocean models and one for space, and 2.6 million were allocated on 2 October 2026.
How results are published
Each study writes down its pass lines, the results that would count as a yes, before its first run, and its live status page shows every run, score and cost as they come in. A finished paper goes out as a preprint on Zenodo with a DOI and then to a journal. The first two, on station-keeping before retirement and on false alarms in maneuver detection, were deposited on 22 September 2026 and are under review at Acta Astronautica. A laboratory notebook with every design decision, submission and result is kept at ai.theinformed.org/research, behind a member login.