Hi — I'm Sean Egan, a Navy information warfare officer and research scientist. This is the foundations book of the set, and it makes one argument the whole way through: a language model is not a mind that happens to use math. It is a geometry, and everything we use it for — similarity, retrieval, reranking, surprise — is a measurement taken inside that space.
Learn the space first and the tricks stop being tricks. Start at Chapter 1; it assumes arithmetic and nothing else. If you have read the LLM book and want the part nobody explains, go straight to Chapter 3.
What you need first
Arithmetic, and a willingness to look at an equation for a few seconds. That is the floor. Every symbol is defined the first time it appears in a chapter, even if an earlier chapter already defined it, and every worked example uses numbers small enough to check on the back of an envelope.
If you have read the other books in this set, Part I will feel familiar for about a page and then go somewhere they do not. Chapters 3 and 4 are the reason this book exists: the behavior of many-dimensional space, and how to read a picture of it without fooling yourself. Neither is covered anywhere else in the family.
Part I
Space Itself
No machine learning yet. Four chapters on the space every model lives in, built out of arrows you can draw on paper — and one chapter that breaks the drawing on purpose.
-
1Numbers as ArrowsA place, a heading, and one number for how much two of them agree12 min
2Distance and AngleTwo rulers for the same pair of points, and how to choose between them11 min
3The Weirdness of Many DimensionsConcentration of measure: where flat-land intuition stops being true13 min
4Projections and ShadowsChoosing the flashlight angle, and what a two-dimensional picture is allowed to prove11 min
What is coming
Part I is written and is complete in itself — it is the whole instrument kit, taught with no machine learning in it at all. The three parts that build on it are being written in order. They are listed here as plans, not as links, because an unwritten chapter should not have a door.
- Part I · Space Itself Numbers as arrows, distance and angle, the weirdness of many dimensions, projections and shadows. Published — four chapters
- Part II · Meaning as Position Where a word's position comes from: co-occurrence counts worked by hand, arithmetic on meaning and where it is folklore, contrastive training read as forces pulling points into clusters, and what it takes to search a map of millions. In writing
- Part III · Geometry Inside the Model Context moving a point, the residual stream as one journey, features as directions, and the probability surface read as surprise. Planned
- Part IV · Instruments and Invention What one point cannot carry and why rerankers read both texts together, the instruments as they are used in a working system, and the open questions at the frontier with the experiments you could run yourself. Planned
The foundations book of five: this book · AI/ML from the Ground Up · Training a Model · How LLMs Actually Work · Diffusion (images).