Functions and Curves

A bold claim about a trillion-dollar machine

Ask an engineer who builds these systems what a large language model is, at bottom, and a surprising number of them will shrug and say: "It's one very large function." That sounds either profound or like a dodge. By the end of this chapter it will be neither — it will be the most honest one-line description of an LLM there is, and you will know exactly what it means.

To get there we need four things, and only four. First, what a function actually is — a machine. Second, how a graph shows a machine's entire behavior at a glance. Third, two celebrity curves this whole site leans on, $e^x$ and $\ln x$. Fourth, a first "squashing" curve, $\sigma(z)$. Those four shapes are the complete visual vocabulary for everything that follows — no genuinely new curve shape appears after this chapter, only new arrangements of these.

Functions are machines

Forget every equation you were ever asked to "solve." A function is a machine. It has an input slot and an output slot. You feed a number into the slot, the machine applies a frozen rule, and a number drops out the other end. Same input, same output, every single time — no memory of the last number, no mood, no surprises.

We write a machine's behavior with the notation $f(x)$, read aloud as "f of x." Here $f$ is the machine's name and $x$ is whatever you choose to feed it. One warning worth a full sentence: $f(x)$ does not mean "$f$ times $x$." The parentheses here mean "fed into" — a genuinely confusing convention that trips up nearly everyone once.

Let us build the house machine we will carry through the whole chapter: $f(x) = 2x + 1$. It doubles whatever you give it, then adds one. Feed it $3$:

Here is the move that makes machines powerful: they chain. The number that drops out of one machine can be fed straight into the next. Introduce a second machine, $g(x) = x^2$, read "g of x equals x squared" — it multiplies $x$ by itself. Now watch what happens when we run our two machines back to back, in each order:

In words: feeding $3$ through $f$ first and then $g$ gives $49$; feeding it through $g$ first and then $f$ gives $19$. Order matters — feeding forward through machines is not shuffling-safe.

Hold that picture, because it is the entire architectural idea of a deep neural network, previewed in nine words: machines feeding machines, hundreds of stages deep. We will harvest that sentence at the end of the chapter.

A graph is behavior you can see

In Chapter 1 we met the 2D plane and $(x, y)$ coordinates. That is all we need to draw a machine. Put the input $x$ on the horizontal axis and draw each output as a height above it. Plot the four points from our table — $(0,1)$, $(1,3)$, $(2,5)$, $(3,7)$ — then imagine filling in every input in between. All of them together trace a single solid line.

Now read the line's behavior straight off its shape. The line $f(x) = 2x + 1$ climbs at exactly the same rate everywhere: every $+1$ of input buys precisely $+2$ of output, whether you are standing at $x = 0$ or at $x = 1000$. A straight line is a machine with the same attitude everywhere.

Contrast that with the squaring machine $g(x) = x^2$. Its table for a spread of inputs:

Those points trace the parabola in the right panel above. Read its behavior: it falls on the left, flattens as it approaches the middle, bottoms out at $x = 0$, then rises on the right. A curve is a machine whose attitude changes from place to place. Hold onto this exact parabola: Chapter 5 spends its entire length standing on it, asking one sharp question — "how steep is it right here?"

And that names the only graph-reading skill this whole site ever needs. When you can look at a picture and say "steep here, flat there, bottoms out over there" without computing a thing, you are reading a graph as behavior. Practice it on every curve below.

$e^x$: growth by multiplying

Start with plain doubling, base $2$: $2^0 = 1$, then $2^1 = 2$, $2^2 = 4$, $2^3 = 8$, and so on. Keep going and by $2^{10}$ you have reached $1{,}024$. Notice the difference from the straight line. The line grows by adding the same amount each step; this grows by multiplying by the same factor each step. That multiply-per-step behavior is what the word exponential actually means.

Doubling is fine, but mathematics has a favorite base, a constant $e \approx 2.718$. Why this particular ungainly number? One teaser, and no more, because the reader has no derivative yet: the curve $e^x$ climbs at a rate exactly equal to its own current height. We will measure that for ourselves in Chapter 5, right after the shrinking-secant trick lands; for now, file it as the reason $e$ is special.

Here is a value table for $e^x$ you can lean on for the rest of the chapter:

Two properties are worth landing hard. First, the output is always positive: feed $e^x$ the number $-100$ and you get a tiny positive sliver, never zero and never negative. Second, equal steps in the input multiply the output by equal factors — one step to the right is always one multiply by $e$:

In words: adding $1$ to the input multiplies the output by $e$. Verify it by hand: $e^1 \approx 2.72$, and $2.72 \times 2.72 \approx 7.39 = e^2$ ✓.

Why does this site care? Because "turn any score — positive or negative — into a positive number, with no ties at zero" is exactly the move a model uses to manufacture probabilities later on (that machine is called softmax; we only name it here). Full payoff in the final section.

$\ln x$: the question machine

Every machine that never repeats an output has an "undo" machine, and $e^x$'s undo is the natural log, written $\ln x$ and read "the natural log of x." It is $e^x$ run backwards: where $e^x$ answers "start at $1$ and multiply by $e$ this many times, what do I get?", $\ln x$ answers the reverse question — "$e$ to the what gives $x$?" Its anchor values are the previous table, read right-to-left: $\ln 1 = 0$ (because $e^0 = 1$), $\ln e = 1$, and $\ln 7.39 \approx 2$ (because $e^2 \approx 7.39$ — the very same fact, reversed).

Because each machine undoes the other, running one and then the other lands you exactly where you started:

In words: take the log of $x$ then feed it to $e^x$, and you are back at $x$; go the other way and the same thing happens. Each machine perfectly undoes the other.

Visually, that partnership is a mirror: the graph of $\ln x$ is the graph of $e^x$ flipped across the $45^\circ$ diagonal, as the figure shows. But the log's real usefulness lives in territory the exponential rarely visits — the inputs below $1$:

Notice the pattern: each time the input is divided by $10$, the output subtracts the same $\approx 2.3$. The log takes "how close to zero is this?" — an awkward question about microscopic numbers — and answers it with an ordinary-sized negative number.

Plant a flag here for later: those negative numbers will literally be the model's exam scores. A model that gave the correct next word a probability of only $0.1$ gets charged a loss of $2.3$. Remember that $2.3$ — it returns in Chapter 12 wearing a suit.

One more property earns the log its keep. It turns multiplying into adding:

In words: the log of a product is the sum of the logs. Hand-check it: $\ln 2 \approx 0.69$, and $\ln 4 = \ln(2 \cdot 2) \approx 1.39 \approx 0.69 + 0.69$ ✓.

Why anyone cares: an LLM multiplies long chains of probabilities together — tiny $\times$ tiny $\times$ tiny — and those products underflow to nonsense fast. Take the log and every one of those multiplications becomes a manageable sum.

$\sigma(z)$: the squash

Raw scores inside a model roam the entire number line — you will see values like $-14.2$, $0.3$, and $88$ side by side. But we often need an answer that lives between $0$ and $1$: belief-sized, percentage-sized. So we need a machine that takes the whole unbounded number line and squashes it into that narrow strip.

That machine is the sigmoid. Here $\sigma$ is the Greek letter sigma, and the input letter is $z$ by long tradition:

In words: take $e$ to the minus $z$, add $1$, and divide $1$ by the result.

Work three checkpoints using the $e$-values you already own. At $z = 0$, we get $e^{-0} = 1$, so $\sigma(0) = \frac{1}{1+1} = 0.5$ — dead center. At $z = 2$, in full:

Now read the curve as behavior, the skill from earlier in this chapter. It is an S-shape that hugs $0$ on the far left, hugs $1$ on the far right, never quite touches either, and is steepest right at the middle point $(0, 0.5)$. Extreme inputs get flattened almost to the rails; all the interesting action happens near zero.

This is only the first member of a whole family of squashing functions. The companion site tours the relatives — tanh, ReLU, and friends — and in Chapter 9 you will meet softmax, the many-options sibling built from the same $e^x$, which turns an entire list of scores into a set of probabilities. Names only for now; each earns its own visit later.

Play: one grapher, five personalities

Everything above sits in one grapher below. Pick any of the five machines from this chapter, then drag horizontally anywhere on the plot to set the input $x$. A point rides the curve while dashed guides drop to both axes — the blue marker shows the input on the $x$-axis, the mint marker shows the output on the $y$-axis, and the readout spells out the number. The same drag reveals five completely different personalities.

Why the LLM cares

Now collect on the opening claim. An LLM is one colossal composed function. Text — turned into numbers in Part II — goes in one end; a spread of beliefs about the next word comes out the other; and in between there is nothing but machines feeding machines, millions of small $f$'s and $g$'s chained exactly like our $g(f(3))$. That is not a metaphor. It is the literal architecture.

Each curve you just met has a job in that machine:

• $e^x$ manufactures positive numbers out of arbitrary scores, so they can become probabilities (softmax, Chapter 9).
• $\ln$ measures how surprised the model was by the right answer — the loss (Part III; that $2.3$ again).
• $\sigma$ and its relatives do the squashing inside the layers.
• The straight line $f(x) = 2x + 1$ is the one-input cousin of Chapter 3's matrices: a matrix transforms many inputs the way $2x + 1$ transforms one — steadily, with no bends.

Which raises the obvious question: why bother with the bends at all? Because chaining nothing but straight-line machines gives you back one big straight-line machine, and no amount of stacking changes that. It is the curves, stirred in between the matrices, that let a network learn behavior more interesting than a straight line. The companion site develops that point properly; for us it is one sentence worth remembering.

So here is the frame to close on. Four shapes: the line, the parabola-style curve, the exponential-and-log pair, and the squash. You now own the complete visual vocabulary of this site. Nothing later introduces a genuinely new curve shape — only new arrangements of these four.

Where we're headed

We can now draw a machine's whole behavior as a curve. The next question is sharper and more local: standing at one single point on a curve — the parabola, say — which way is downhill, and how steep is it right here? That tiny question has a precise answer called the derivative, and it is no footnote: it is the engine that trains every neural network on Earth. Chapter 5 builds it with nothing but subtraction and division.