The Math Toolkit

Five tools, then we build

Last chapter we landed one sentence: machine learning is fitting a function to data. To actually do that, a machine needs to hold data, compare two pieces of it, transform it, find its way downhill on a landscape of wrongness, and reason about what it is not sure of. That is exactly five mathematical tools — and this chapter forges all five, each with one picture and one worked example you can check by hand.

Let me set expectations honestly and warmly: this is a fast, standalone toolkit, not a math course. If you have read the sister site's Part I — How LLMs Actually Work — you can skim, because this is the condensed version, and each tool links there for the slow build. If you have not, everything you need is right here. Here is the whole shelf at once.

Read the shelf left to right, in the order of the jobs. Vectors hold our inputs. The dot product compares two of them. Matrices transform them. Derivatives tell us which way is downhill when we are trying to be less wrong. And probability lets a model spread belief across outcomes instead of pretending to be certain. Every model in this book — from a straight line to a chatbot — is these five, arranged.

One promise to reduce any dread: no tool here is harder than one-digit arithmetic done a few times. We keep every example in two dimensions with small whole numbers, because the pictures stop working past three dimensions but the arithmetic never changes — it only gets longer.

Tool 1 — Vectors: arrows and lists

Picture it before any symbols. A vector is a list of numbers that is also an arrow. Think of a walking direction: "3 blocks east, 2 blocks north" is one instruction carrying two numbers, and it has a built-in direction and distance. Draw it as an arrow from where you stand — the origin — to the point 3 across and 2 up, and you have drawn a vector.

Notation, kept to the house rules: vectors are written bold and lowercase, like the vector $\mathbf{v}$, and its two numbers are its components, $v_1$ and $v_2$. Our running arrow this whole chapter is $\mathbf{v} = [3, 2]$. In every figure and interactive below it is drawn in mint, because the input we care about is always mint on this site. We will also carry a contrast arrow, $\mathbf{u} = [1, 3]$, always in amber.

There are two things you can do to a vector, and both are just "do it slot by slot." Adding two vectors adds their matching components — on the page, that is walking one trip and then setting off on the second from wherever the first left you, tip-to-tail.

In words: line up the two lists, add the top numbers to get 4, add the bottom numbers to get 5, and the answer is the vector [4, 5] — one trip that lands where the two trips end up.

Scaling is multiplying a vector by a plain single number — a scalar — which stretches every component at once.

In words: double each number, so [3, 2] becomes [6, 4] — an arrow twice as long pointing exactly the same way. A positive scalar never changes the direction, which is why direction alone can carry information.

One more thing an arrow has, as opposed to something you do to it: a length. It is the property the rest of this book leans on hardest. Our arrow $\mathbf{v} = [3, 2]$ says "3 blocks east, 2 blocks north" — but a bird flying from the origin to the tip does not follow the streets, it cuts the corner. How far does it fly? That is the oldest trick in geometry, the one you half-remember: the two components are the short sides of a right triangle and the arrow is the long side, so square each component, add them, and take the square root.

In words: 3 squared is 9, 2 squared is 4, together they make 13, and the square root of 13 is about 3.61 — the arrow reaches about 3.61 blocks as the crow flies. The double bars around $\mathbf{v}$ are how we write "the length of v."

The recipe never changes no matter how many components there are: square every slot, add them all up, take the square root. Our amber contrast arrow $\mathbf{u} = [1, 3]$ comes out at $\lVert \mathbf{u} \rVert = \sqrt{1 + 9} = \sqrt{10} \approx 3.16$ blocks, a touch shorter than the mint one. Hold on to both numbers — we spend them on the next page, and the double bars turn up again every time a model cares how big something is rather than which way it points.

Now the liberating leap, in one sentence: a vector with $d$ components is an arrow in $d$-dimensional space that nobody can picture, and you never need to — addition is still "add matching slots," scaling is still "multiply every slot." Real models hold each input as a vector with thousands of components; we always draw the flat 2D shadow and trust the arithmetic for the rest. Take the controls below — drag the arrows and watch the numbers move. For the full, slow build, the sister site's Numbers That Point is the same idea told slowly.

Tool 2 — The dot product: one number for agreement

Over and over, a model needs to ask one question: how much do these two things agree? If our inputs are arrows, that question becomes "do these two arrows point the same way?" The dot product answers it with a single number.

The recipe, in words before symbols: multiply matching components, then add. It is written with a center dot, $\mathbf{v} \cdot \mathbf{u}$, read "v dot u." On our running pair, every digit shown:

In words: multiply the top numbers (3 times 1 is 3), multiply the bottom numbers (2 times 3 is 6), add those together, and get 9. A positive answer means the arrows agree — they point the same general way.

Positive is only one of three verdicts. Let us test the promise on the other two.

Why does "multiply matching slots and add" measure agreement? The honest intuition: each pair of matching components casts a vote. Same sign, positive vote — the two arrows push the same way along that axis. Opposite signs, negative vote. Big aligned components, big votes. The dot product tallies the whole election into one number, and the sign of the total tells you who won.

One refinement, and the lengths we just measured are what make it possible. The raw dot product grows with length as well as with alignment, so to compare pure directions we divide out both arrows' lengths and get cosine similarity, a score locked between −1 and +1. We have both lengths already, so we can finish the job here: the dot product 9, divided by $3.61 \times 3.16 \approx 11.4$, is about 0.79 — the two arrows point the same general way, and that is a number you worked out rather than one I handed you. The full treatment (projection, the exact geometry, why the answer is a cosine at all) is the sister site's Measuring Similarity. Notice, in prose: the general form of the recipe is just the two products added, $\mathbf{v} \cdot \mathbf{u} = v_1 u_1 + v_2 u_2$.

Tool 3 — Matrices: machines that transform vectors

Picture before symbols again. A matrix is a grid of numbers that is also a machine: feed it a vector, it hands back a new vector. This is where a model's learned numbers live. Our star matrix is $\mathbf{W} = \left[\begin{smallmatrix} 1 & 2 \\ 3 & 0 \end{smallmatrix}\right]$. In the figures below it is drawn in violet, because weights and parameters are always violet on this site.

In words: a two-by-two grid — two rows, two columns, rows named first. The four numbers are the machine's settings.

Notation, per the house rules: matrices are bold and uppercase, like the matrix $\mathbf{W}$, and we address an entry by row then column — the entry in row $i$, column $j$. Real models use grids like 4096 by 4096; that changes only your patience, never the procedure.

And the procedure runs on Tool 2. Feed $\mathbf{W}$ the mint vector $\mathbf{v} = [3, 2]$. Each row of $\mathbf{W}$ takes a dot product with the input: row 1's dot product becomes output slot 1, row 2's becomes output slot 2.

In words: row 1 is [1, 2], and [1, 2] dot [3, 2] is 3 plus 4, which is 7. Row 2 is [3, 0], and [3, 0] dot [3, 2] is 9 plus 0, which is 9. Stack the answers and the output vector is [7, 9]. A matrix is just dot products in bulk — one per row.

There is a second way to read that same arithmetic, and it names the tool. Each row is a recipe. Output slot 1 is "1 part of the input's first number, 2 parts of its second"; row 2 is "3 parts of the first, none of the second." Every output number is a weighted blend of all the inputs, and the matrix entries are the blend proportions. That is why they are called weights — and when a model "learns," these proportions are exactly what change.

Zoom out and the machine has a geometric face too: feed a matrix not one vector but the whole plane, and it warps the grid — stretching, leaning, rotating space — while straight lines stay straight and the origin stays put. Different entries, different machine. The slow build (the three faces of a matrix, rotation, the columns trick) is the sister site's Grids That Transform.

Tool 4 — Derivatives: which way is downhill

Training means making a model less wrong, and in a few chapters "wrongness" will become a literal landscape we want to walk to the bottom of. To walk downhill in thick fog you need only one thing: the tilt of the ground right under your feet. That local tilt is the derivative.

First, the word slope, because everything after this hangs on it. On a straight ramp, the slope is how much you rise for every step you take forward: walk one pace to the right and go up two, and the slope is 2 — rise over run, two over one. Uphill counts as positive, downhill as negative, and flat ground is slope zero. That is the whole word. The only new thing a curve adds is that the steepness changes as you walk along it.

Picture it on the site's house curve, the parabola $f(x) = x^2$. On a curve the slope depends on where you stand — gentle near the bottom, steep out on the arms. The derivative, written $\frac{df}{dx}$ and read "the derivative of f with respect to x," is that slope at a single point. For this parabola the pattern is short enough to state in one line — and then, because I am not going to ask you to take it on faith, to check by hand:

In words: the slope at any point is twice the position. At $x = 1$ the slope is 2, at $x = 3$ it is 6, and at $x = 0$ it is 0 — and flat, slope zero, is exactly where a valley bottoms out.

Now turn that tilt into a walk — this is the single rule behind all of training. Step opposite the slope: your new position is your old position minus a small step-size times the slope. That step-size is the learning rate, written $\eta$ — the Greek letter eta.

In words: to get your next position, take the slope where you stand, shrink it by the learning rate, and subtract it — the minus sign is what makes you go downhill instead of up.

Scale it up in one sentence. A real model has not one dial but millions; stack the slope for every dial into one long vector called the gradient, written $\nabla L$ with the nabla symbol and read "the gradient of the loss" — "loss" being the name we will give that landscape of wrongness next chapter — and step against it component by component. Two dials or two billion, the walk is identical. The slow build (measuring a slope with shrinking secants, learning-rate pathologies, the gradient in two dimensions) is the sister site's Slopes and Descent.

Tool 5 — Probability: belief spread across outcomes

A good model rarely gets to be certain. Asked "cat or dog?" about a blurry photo, the honest answer is not a guess but a spread: "80 percent cat, 20 percent dog." Probability is the language for spreading belief instead of faking certainty.

Make it concrete and countable. A bag holds 10 marbles — 5 mint, 3 amber, 2 violet — and you draw one without looking. The probability of drawing mint is the share of mint marbles:

In words: five of the ten marbles are mint, so the chance of mint is five-tenths, or 0.5. The same counting gives $p(\text{amber}) = 0.3$ and $p(\text{violet}) = 0.2$.

There are two house rules, and it helps to frame them as a conservation law. Every probability sits between 0 and 1, and across all the outcomes they add up to exactly 1. You own one single unit of belief, and you may only divide it among the outcomes — never mint more.

In words: add the three shares — a half plus three-tenths plus two-tenths — and they land on exactly 1, one whole unit of belief accounted for.

The full list of outcomes with their shares is a probability distribution, and its natural portrait is a bar chart whose heights sum to 1. Two more ideas belong here in a sentence each, because later chapters lean on them. First, belief has strings attached: the probability of something given what you already know — written with a vertical bar and read "given" — can differ completely from the plain probability, and that is conditional probability. Second, a distribution can be turned into one concrete choice by sampling — rolling a weighted die whose faces match the bars. The slow build (shapes of belief, conditioning, the lumpiness of small samples) is the sister site's Chance and Choice. One forward glance: Chapter 4 will hand a model a way to turn a raw score into a probability like these, and multi-class prediction will spread belief across many bars at once — you have now met every ingredient it uses.

Where you'll meet this

Let me lay these five tools onto the road immediately ahead, so none of them feels like homework for its own sake. It is a single connected route.

Next chapter, Fitting a Line, holds each data point as a small vector, measures wrongness as a landscape, and uses the derivative to walk downhill to the best line — Tools 1 and 4 doing the first real machine learning you will see happen. Chapter 4, Drawing Boundaries, computes a score as a dot product (Tool 2) and turns it into a probability (Tool 5) to decide "which class." Then Part II's artificial neuron is a dot product followed by a squash; deep networks are matrices (Tool 3) stacked with gentle bends between them, and the whole stack is trained by gradient descent (Tool 4). The tools do not get swapped out — they get combined.

Here is the payoff, so you trust the investment: with these five, you can read the equation of almost any model in this site, because almost every model is some arrangement of "multiply by a matrix, bend, compare with a dot product, turn into a probability, and adjust by walking downhill." There is genuinely not a sixth secret tool waiting in Part III. So keep this chapter as your reference shelf: when a later equation looks scary, it is one of these five wearing more subscripts — and if any tool felt too fast, the sister-site link beside it is the same idea told slowly.