Numbers That Point

An arrow and a list

You text a friend directions to a coffee shop: from the station, go 3 blocks east and 2 blocks north. That is one instruction, and it is made of exactly two numbers — yet it already carries a direction (roughly northeast) and a distance (a little more than three blocks). Those two numbers are an arrow.

That double life — a short list of numbers that is also an arrow — is the single most-used object on this entire site. So we are going to start here, from zero. Part I is six short chapters, each one small honest idea at a time; by Part II these same arrows will literally carry the meaning of words that a language model reads.

The object has a name. A vector is a list of numbers that we treat as one single thing — and, equivalently, an arrow with a direction and a length. Both views are always available at once, and we will swap between them constantly. Let's meet it properly.

What a vector is

Before any symbols, look at the picture. Here is a flat 2D plane — mathematicians call this plane $\mathbb{R}^2$, which you can just read as "the 2D plane," and that is all the fancy notation we need for now. On it, one arrow starts at the corner (the origin) and reaches out to the point 3-across and 2-up.

Now the notation, which we will keep for the whole site. We write a vector as a bold lowercase letter, $\mathbf{v}$, read aloud as "the vector v." We write its two numbers stacked in a column:

In words: the vector v is the pair of numbers 3 and 2 — the first is how far to go across, the second is how far to go up.

The individual numbers are the vector's components, and we pick them out with subscripts: $v_1 = 3$ is the across-part and $v_2 = 2$ is the up-part, read aloud as "v-one, v-two." That is the whole grammar of a vector — a name in bold, and its components numbered underneath.

Notice that both a direction and a length are already baked in. By eye, this arrow is a little longer than 3 units and points east-north-east. We will compute lengths exactly in the next chapter, where length pairs naturally with a new operation — so resist reaching for Pythagoras here; it belongs there, not yet.

Adding arrows

Once you can draw one arrow, the natural question is what happens when you do two trips in a row. Picture it first: walk $\mathbf{v} = [3, 2]$ (the mint arrow), and then, from wherever you stopped, walk a second vector $\mathbf{u} = [1, 3]$ (the amber arrow). Where do you end up? 4 across and 5 up. That single combined trip — origin straight to the finish — is the sum, drawn in violet.

The arithmetic is exactly as gentle as the picture. To add two vectors, you add the matching components: acrosses with acrosses, ups with ups.

In words: add the two first-components to get the new first component (3 plus 1 is 4), and add the two second-components to get the new second component (2 plus 3 is 5). Nothing mixes across the slots.

One more thing worth noticing: the order does not matter. Walk u first and then v, and you land at the very same corner — the two routes trace the two sides of the same parallelogram. Addition is symmetric like that, all the way through.

Stretching and flipping

The other basic move is to multiply a vector by an ordinary number, which stretches or shrinks it. When you do, every component gets multiplied. Double our vector and it becomes twice as long in the same direction; halve it and it becomes half as long, still in the same direction.

In words: to double a vector, double each of its numbers — 3 becomes 6 and 2 becomes 4. Similarly $0.5\,\mathbf{v} = [1.5, 1]$: half of each number.

Here is the key observation, worth its own sentence: scaling by a positive number never changes the direction — it only changes the length. Direction and length are separately controllable knobs. That separation is exactly why "direction" can someday carry information on its own, independent of how long an arrow happens to be.

And a negative number? Then the arrow flips: $-\mathbf{v} = [-3, -2]$ points in exactly the opposite direction, same slope, other way. A single minus sign reverses the whole trip. Hold that thought — in the next chapter, "opposite" becomes a measurable kind of disagreement between two arrows.

A quick piece of vocabulary. The plain number you multiply by — the 2, or the 0.5, or the −1 — is called a scalar in this setting: literally, the thing that scales. We write scalars as plain italic lowercase letters, to keep them visually distinct from the bold vectors they act on.

See it move

Two ideas are much easier to feel in motion than to read: that the dashed component lines are the list of numbers, and that scaling slides the tip along a fixed direction line without ever rotating it. The short video makes both happen in front of you.

Now take the controls yourself. The playground below lets you drag an arrow's tip and watch its list of numbers update in real time, switch to tip-to-tail addition, and slide a scalar up and down — including through zero into negative territory.

From two numbers to thousands

Everything so far lived on a flat page. But nothing about the arithmetic was tied to two numbers. A 3D vector is a list of three numbers — [3, 2, 5] — an arrow in a room instead of on a page. You can still just about picture it: two of the numbers place you on the floor, the third lifts you off it.

Now the honest leap. A vector with $d$ numbers is an arrow in a $d$-dimensional space that nobody can picture — and here is the liberating secret: you don't need to. Addition is still "add matching slots." Scaling is still "multiply every slot." The arithmetic never gets harder as the list grows; it only gets longer. We write $d$ for the number of dimensions — read "d, the number of dimensions" — and real language models use vectors with $d$ in the thousands. Throughout this site we will draw the 2D shadow of an idea and trust the arithmetic to carry it up the ladder. (2D teaching examples first, always — that is a rule of the house.)

Why the LLM cares

This is why we bothered. Inside a language model, every token — every chunk of text it reads — is stored as a vector, typically one with thousands of components. When you hear that a model has "embeddings," those embeddings are exactly these arrows: long lists of numbers, each one pointing somewhere in a space too big to draw.

Here is the payoff we are working toward, previewed but not yet earned: the direction a token's arrow points will encode its meaning. The arrows for "cat" and "dog" will end up pointing nearly the same way, while "cat" and "bicycle" point off in unrelated directions. We build that result carefully in Chapter 8, Meaning as Direction; everything between here and there is learning to compute with these arrows.

And you have already met, in miniature, the operations that run billions of times per sentence. Adding vectors — the tip-to-tail walk from a moment ago — is literally how a transformer merges information from different words. Scaling is how it turns each influence up or down. The machinery is enormous; the moves are the two you just learned by dragging an arrow around a grid.