Which of these two is closer?
There are two rulers for closeness, one for the gap between points and one for the angle between headings, and this chapter builds both.
We need both because the dot product mixes them. It is alignment multiplied by length multiplied by length, so a big score can mean "these point the same way" or "one of these is enormous," and the number alone will not say which. That matters, because the closeness question is the one a working system asks all day. Which stored memory is nearest this one? Which document best matches this query? Which of last week's sentences most resembles the sentence he just said?
The first ruler: the gap between two points
The most ordinary answer to "how far apart" is the one a tape measure gives: the straight-line gap. To get it, walk from one point to the other and measure the trip.
The trip from u to v is the difference vector $\mathbf{v} - \mathbf{u}$, and its length is the distance we want. Length comes from Pythagoras: square each entry, add, take the square root.
$$\|\mathbf{v} - \mathbf{u}\| = \sqrt{(v_1 - u_1)^2 + (v_2 - u_2)^2}$$In words: subtract matching entries to get the trip from one point to the other, square each of those differences, add them up, and take the square root. That is the straight-line distance, what people mean by Euclidean distance.
The gap, by hand
Our running pair is still $\mathbf{v} = [3, 2]$ and $\mathbf{u} = [1, 3]$. Subtract entry by entry: $3 - 1 = 2$ across, and $2 - 3 = -1$ up, so the trip from u to v is $[2, -1]$, two to the right and one down.
Now its length: $2^2 + (-1)^2 = 4 + 1 = 5$, and the square root of 5 is about 2.24.
The minus sign vanished when we squared it. Distance does not care which of the two points you started from. The trip the other way, $[-2, 1]$, has the same length.
In words: these two points sit about 2.24 units apart, and that number would be the same if we had measured from v to u instead.
This ruler is familiar and often right. For most of what models do it is the wrong one, and the figure below shows why.
The second ruler: the angle between two headings
Take u and stretch it to three times its length: $3\mathbf{u} = [3, 9]$. It has not turned at all. It lies along the same line, as the figure shows. But the gap from v has gone from 2.24 to 7.00, because $\mathbf{v} - 3\mathbf{u} = [0, -7]$.
By the first ruler, v and u were close and v and 3u are distant. By any sensible reading of the picture, u and 3u are the same direction and should score identically against v. The second ruler is already in hand. Rearrange the shadow formula from the last chapter to put the angle on its own:
$$\cos\theta = \frac{\mathbf{v} \cdot \mathbf{u}}{\|\mathbf{v}\| \, \|\mathbf{u}\|}$$In words: take the dot product and divide out both lengths. What is left is pure alignment, a number running from +1 for identical headings, through 0 at a right angle, to −1 for dead opposite. This is called cosine similarity.
The heading by hand, and what stretching does to it
We know $\mathbf{v} \cdot \mathbf{u} = 9$ from the last chapter, with $\|\mathbf{v}\| = \sqrt{13} \approx 3.61$ and $\|\mathbf{u}\| = \sqrt{10} \approx 3.16$. So the alignment score is $9 \div (3.61 \times 3.16) \approx 0.79$.
Now do it again with the stretched arrow. The dot product triples: $\mathbf{v} \cdot 3\mathbf{u} = 3 \times 3 + 2 \times 9 = 9 + 18 = 27$. But so does the length: $\|3\mathbf{u}\| = \sqrt{9 + 81} = \sqrt{90} \approx 9.49$, which is $3 \times 3.16$. Divide: $27 \div (3.61 \times 9.49) \approx 0.79$.
In words: the same 0.79. The factor of three appeared on the top and on the bottom and canceled itself out. That cancellation is the point of dividing by the lengths.
So you have a choice to make, and it is a question about the data rather than about taste: does length carry information here, or is it noise?
In many systems, length is noise. A long document and a short one about the same subject produce vectors of different sizes for reasons that have nothing to do with subject. A word that appears constantly gets a bigger vector than a rare one, and "common" is not "meaningful." There, length is telling you about the container rather than the contents, and dividing it out removes a known distortion.
In other systems length is the signal. Inside an attention head, a token that should dominate the sentence gets to shout, and the raw dot product lets it. Strip the length out there and you have thrown away the model's way of saying "this one matters more."
Put everyone on the same sphere
Instead of adjusting the score afterward, move the points first. This is the picture the rest of the book runs on.
To normalize a vector, divide it by its own length. The result is a unit vector, the hat notation from the last chapter, pointing the same way with its length set to 1.
$$\hat{\mathbf{v}} = \frac{\mathbf{v}}{\|\mathbf{v}\|}$$In words: divide every entry of v by v's own length. The arrow keeps its heading and loses its size, which parks its tip on the circle of radius 1 around the origin.
Normalizing our pair
$\|\mathbf{v}\| = \sqrt{13} \approx 3.606$, so $\hat{\mathbf{v}} = [3 \div 3.606,\; 2 \div 3.606] \approx [0.832,\; 0.555]$. Check it: $0.832^2 + 0.555^2 \approx 0.692 + 0.308 = 1.000$. Length 1, as promised.
Likewise $\|\mathbf{u}\| = \sqrt{10} \approx 3.162$, so $\hat{\mathbf{u}} \approx [0.316,\; 0.949]$.
And the dot product of the two unit arrows: $0.832 \times 0.316 + 0.555 \times 0.949 \approx 0.263 + 0.527 = 0.790$, the alignment score itself, with no division left to do.
In words: once both arrows sit on the circle, the plain dot product is the alignment score. Normalizing does the dividing once, up front, instead of every time you compare.
That is why real systems store normalized vectors. A search index holding a million passages can normalize all of them once, when they are written, and then answer every future query with a bare dot product, the cheapest operation a computer owns.
On the sphere, the two rulers agree
Once both points sit on the unit circle, the two rulers give the same ranking, always. Most people who use these tools have never been shown this.
Work out the squared gap between two unit arrows and watch what survives:
$$\|\hat{\mathbf{v}} - \hat{\mathbf{u}}\|^2 = 2 - 2\cos\theta$$In words: the squared straight-line distance between two points on the unit circle is 2 minus twice their alignment score. Distance and alignment are one quantity, read with opposite signs.
Three checkpoints, all hand-checkable
Take $\hat{\mathbf{a}} = [1, 0]$ and swing a partner around the circle.
Same heading, partner $[1, 0]$: alignment is 1, so the formula says the squared gap is $2 - 2 = 0$. The two points are the same point, gap 0.
Right angle, partner $[0, 1]$: alignment is 0, so the squared gap is $2 - 0 = 2$, and the gap is the square root of 2, about 1.41. Check it directly. The trip between them is $[-1, 1]$, whose length is $\sqrt{1 + 1} = \sqrt{2}$. It matches.
Dead opposite, partner $[-1, 0]$: alignment is −1, so the squared gap is $2 + 2 = 4$ and the gap is 2, straight across a circle of radius 1, which is the diameter. It matches.
In words: as the alignment score slides from +1 down to −1, the gap climbs steadily from 0 up to 2, and it never doubles back. Rank by one and you have ranked by the other.
See it move
The scale mode below runs this chapter's experiment with your own hands. Drag the amber arrow to a heading you like, then run the scale slider up and down. Watch the list of numbers change while the heading refuses to.
Where this shows up
- Every vector database. When a system stores embeddings for later search, it almost always normalizes them on the way in. That single decision is what lets the search itself be nothing but dot products. RAG and Vector Search in the LLM book builds the retrieval loop on top of this.
- Drift detection. "Something has changed in how this person writes" is a distance between two averaged directions, measured over time. If the vectors were not normalized, the measurement would mostly report that he wrote longer emails this month.
- Deduplication. "Have I already been told this?" is a threshold on one of these two rulers. Because they rank identically on the sphere, a team can argue about which one they are using and still ship the same behavior.
- Clustering. Algorithms that group points, like the k-means routine in the AI/ML book's classical toolbox, are built on a distance. Run them on unnormalized text vectors and the first cluster you find is often just "the long ones."
All four of those live or die on distances measured in a space with many more than two dimensions. We have been drawing on a page because a page is checkable. The next chapter is about what stops being true when we leave it.
What you now know
- Euclidean distance measures the straight-line gap between two points, by subtracting matching entries, squaring, adding and taking the square root.
- Cosine similarity measures only the angle between two headings, because dividing by both lengths cancels any stretching.
- Choosing between them is one question about your data: does magnitude carry information, or is it an artifact of the container?
- Normalizing divides a vector by its own length, parking every point on the unit sphere, after which a plain dot product is already the alignment score.
- On that sphere the squared gap equals 2 minus twice the alignment score, so the two rulers produce the same ranking, and normalizing permanently destroys whatever the magnitudes were carrying.
Where we're headed
Every picture so far has been drawn on a flat page. Real models work in hundreds or thousands of dimensions, and the page lies about what happens there. The next chapter runs the experiment, sampling the same random directions at 2, 10, 100 and 1000 dimensions, and watches the answer change shape.