Skip to the page
HOW MACHINES LEARN
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9

Page 6 of 9

Watch the space take shape

Blame is where a point wants to go. Give every point what it wants, a little at a time, and the kinds come apart.

Press Play. The first picture shows the points as given, tinted by the network's current answer. The second, the inner view, shows where the network has put each point, with one arrow per bin. The thin bright arrows are tugs. When it has finished, choose Bend off and press Start over.

What you are watching

Each adjustment uses every point. Each point pulls the weights toward whatever would move it the right way. The weights are shared by all the points, so they settle on the changes that many points agree about, and the whole cloud drifts. After a few hundred adjustments one straight cut in the inner view separates the kinds.

The arrangement it ends in is what people mean when they say a network has learned a representation. Nobody drew it; it is what a few hundred small tugs left behind.

With the bend off, the inner view is the first picture turned and stretched, because every row is now a plain table. The tugs still pull and the tables give all they can. The kinds stay tangled.

From a position to probabilities

At the bottom of a board, each bin has its own list of weights. Drawn as a picture, each bin has an arrow, and a point's score for a bin is how far the point lies along that bin's arrow.

Scores can be any size and any sign. To turn them into probabilities that add up to 1, raise the number e to each score and divide by the total. Textbooks call this step softmax.

score of a bin        =  the bin's weights × the point's levels, added up
probability of a bin  =  e^score  ÷  (the sum of e^score over all the bins)
blame at a bin        =  probability given − probability wanted

a bin that was given too much probability has positive blame, and the right bin, which was given too little, has negative blame.

Drag the gold point. Each bin's bar is how far the point lies along that bin's arrow, turned into a probability. Switch on Show the tug, then press Follow the tug.

Blame is a direction

Carry the bins' blame up one row, by the rule from the last page, and it lands on the point's own levels. Each bin sends its blame up its own wires, so each bin's arrow arrives multiplied by that bin's blame.

Look at the signs. The right bin's blame is negative and every wrong bin's is positive. Taken together, the blame on the point says: move toward the right bin's arrow, and away from the others, each in proportion to how wrong that bin's probability is. That direction is the tug in the picture above.

A point cannot move by itself. Its position is computed from the weights above it. So those weights are adjusted, each by blame below times level above, and the effect is that the point moves the way it was pulled.

The arithmetic of that move

Take one slot. Its total is the sum of each incoming weight times the level above it. The rule changes each incoming weight by minus the step size, times the slot's blame, times the level above.

change in a weight      =  − step size × blame × level above
change in the total     =  sum of (change in each weight × level above)
                        =  − step size × blame × (sum of the levels above, each squared)

the slot's total moves opposite to its blame, and it moves further when more signal was coming in.

The picture at the top of this page is that rule at work on every point at once.

What each part was doing

The stretching, the shrinking and the folding each have one job, and none of them was chosen for its looks.

PartWhat it does to the pictureWhat it is for
A table of weightsTurns, stretches, shrinks and shears the whole space.Lines the points up so that the next row's gates fall in useful places. Alone it cannot separate tangled kinds.
An offsetSlides the whole picture.Sets where each gate's line falls.
The gateFolds: presses everything negative flat onto a wall.Lets points that were far apart be laid together, and lets tangled kinds come apart. No table can do this.
The last tableMeasures each point along each bin's arrow.Turns a position into one score per bin.
The squashTurns the scores into probabilities that add to 1.Gives the error something to compare with the right answer.
Blame and the adjusting ruleNudges every table so each point moves the way it was pulled.This is the learning. It decides how much to turn, stretch and slide, and so where the folds land.

Why a lesson from one point carries to others

A correction made for one point changes the answer for every point that uses the same wires. Points that resemble each other use nearly the same wires, so they move together. That is why the tinted regions grow as whole areas, and why a network can be right about a point it never saw.

The weights are built out of the examples

The adjusting rule says a weight changes by blame below times level above. For a slot in the first row, the level above is the input itself. So after training, that slot's list of weights equals its starting values plus a sum of the inputs it saw, each scaled by the slot's blame for that input.

This can be checked. On a trained network the two sides agree to fifteen decimal places. In a second check the inputs all lay in a 3-dimensional slice of a 10-dimensional space, and the change in the first-row weights had no measurable part outside that slice. A slot's weights are assembled from the data it was shown.

Inputs that share a feature and call for the same correction push the same weights the same way each time, and those pushes pile up. Pushes that come from one input's quirks point in unrelated directions and mostly cancel. Readers from signals work will recognize averaging. This account, that corrections from similar examples reinforce each other, was proposed in 2020 under the name coherent gradients. It is a hypothesis with experiments behind it.

Chatterjee, Coherent gradients ↗

The inputs learn too

The blame does not stop at the top row of weights. It lands on the input's own numbers. In the eight-weight board on the last page it came to +0.731 on the first number and −0.731 on the second.

In a language model those numbers are stored, one list for every token, and they are stepped downhill like any weight. On the eight-weight board, adjusting the token as well lifts the right bin to 40% in place of 37%. So the lists that stand for words, and the weights that work on them, are shaped together by the one rule.

CARRY THIS FORWARD

Nobody designs the shape. Every example tugs its point toward its own bin, the tables give a little, and the shape is what is left.