Skip to the page
HOW MACHINES LEARN
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9

Page 5 of 9

How a weight deep inside gets corrected

The bottom row has a miss to work from. A weight in the middle does not. That was the unsolved problem for 25 years, and the fix was to let every slot pass an amount.

Drag the gold weight, then change the kind of slot. With all-or-none slots the error sits on a flat shelf and then jumps. With slots that pass an amount it changes smoothly, and a small nudge shows which way is better.

The wall

The rule from page 2 needs a miss for each slot. A slot in the bottom row has one, because the right answer is given. A slot in a middle row has none. Nobody says what its level should have been.

Every learning machine of the 1960s could adjust its last row of weights and nothing above it. In Rosenblatt's Mark I the wiring into the 512 middle slots was random and stayed fixed. Widrow's group worked for years on a machine with two rows of weights and never found a rule that trained both.

Why a ball cannot be taught

Picture a pegboard with a ball dropped in at the top. The ball bounces down one route and lands in a bin. Make one peg slightly larger. The ball still goes the same way, unless it happened to be balanced on that peg, and then the outcome flips. There is no “slightly better” to follow.

The early machines were boards of this kind, because every slot fired all or nothing. The picture at the top of this page is one of them. Drag its gold weight and watch what the error does.

On a flat shelf, rocking a weight shows nothing, and calculus has nothing to work with either. That is the wall.

The way through is to change what travels down the board. Let each slot pass an amount, and an input stops being a ball on one route. It becomes a pattern of amounts spread across each row, with some of it on every wire. Then a small change to any weight shifts every bin's score a little, and there is a direction to follow.

The fix, in its authors' words

That paragraph is the design decision. The unit has to bend, or middle rows add nothing. The bend has to have a slope, or no correction can pass through it. They chose a smooth S-shaped curve, a rounded version of the 1943 step.

Rates multiply along a chain

Take the shortest chain that has a gate in it: one input, a weight, a gate, and a second weight.

input 2  →  × first weight 0.5  →  gate  →  × second weight 0.5  →  prediction 0.5      target 1

slope of the second weight  =  miss × level it carried
                            =  (−0.5) × 1  =  −0.5

slope of the first weight   =  miss × second weight × gate's slope × input
                            =  (−0.5) ×     0.5       ×      1       ×   2    =  −0.5

to find how the first weight affects the error, multiply the rates along the wires between it and the output.

Rates multiply the way unit conversions do. Turning the first weight moves the slot's total 2 for 1. The open gate passes that along 1 for 1. The second weight moves the prediction 0.5 for 1. So the first weight moves the prediction 1 for 1.

Each of the four factors is a condition on the first weight. If there was no miss, it does not change. If the wire from its slot to the output is weak, it changes a little. If the gate was shut, it does not change. If its input was zero, it does not change. Step both weights by a tenth of their slopes and the error falls from 0.125 to 0.078.

The third factor is where the history turned. For an all-or-none slot it is zero everywhere, so the whole product is zero and nothing reaches the first weight. For the 1986 curve it is a number between 0 and 0.25. For the gate it is 1 or 0.

One token through a whole board

The board below has eight weights. An input to a board is called a token from here on, the word language models use. Follow one token down, then follow the correction back up.

The quantity that travels upward is called blame on these pages. A slot's blame is how fast the error would rise if that slot's total went up. Negative blame means the slot should have been higher.

Press Next and read each step. Cyan is signal going down. Amber is blame coming back up. Touch any wire to see its numbers: the weight, the level above it, the blame below it, and the slope those give.

Four weights moved, and all four sit on wires that carried signal for this token. The other four sit on wires out of slot B, which was silent, or into slot B, whose gate was shut.

Each change has a reason you can say aloud. Slot A was on and bin 1 should have been higher, so the weight from A to bin 1 went up. Slot A was on and bin 2 should have been lower, so the weight from A to bin 2 went down.

The rule for any board

going down    level of a slot  =  gate( sum of weight × level above, plus offset )
coming up     blame of a slot  =  (1 if its gate was open, else 0) × sum of weight × blame below
adjusting     weight           =  weight − step size × blame below × level above

signal goes down through the weights, blame comes back up through the same weights, and each weight changes by the product of the two numbers at its ends.

The blame comes back up the same wires because of what a weight does on the way down. A wire with a weight of 3 delivers three times the upper slot's level to the lower slot. So the lower slot's blame counts three times toward the upper slot's.

This procedure is called backpropagation. The whole of it is on this page.

Why backward, and what it saves

There is one error and there are many weights. Starting from the error, the part of the calculation that every weight shares gets done once.

On a network with 6,154 weights and offsets, rocking each one takes 6,155 runs of the network. The backward pass takes one run down and one run up, and gives the same slopes to nine decimal places. In a timing test the rocking took about 4,900 times longer. A model with a billion weights would need a billion and one runs for every example.

The pegboard, corrected

If you picture a network as a pegboard, most of the picture holds. The table marks the parts that need changing.

The pegboardKeep or changeThe network
Rows of pegsKeepRows of slots, with a table of weights between each pair of rows.
Bins at the bottomKeepOne score per bin. The scores become probabilities.
Pegs of adjustable sizeKeepWeights. Each one scales what passes along one wire, and a negative weight subtracts.
A ball dropped in at the topChangeA pattern of amounts across the whole top row.
The ball takes one routeChangeEvery wire carries some of the pattern, and each slot adds up what arrives.
Different balls reach different binsKeepGates open and shut depending on the input, so different inputs use different sets of wires.
A springy row that bounces the ballChangeThe gate acts in place. A positive total passes and a negative total becomes zero. Nothing goes back up while an input is being sorted.

The first change is the one the field made in 1986. During training something does travel back up the board. It is the blame, and it uses the same wires the signal came down.

CARRY THIS FORWARD

Signal goes down through the weights. Blame comes back up through the same weights. Each weight moves by the product of the two numbers at its ends.