So… What Is AI/ML?

What did we actually build?

Nineteen chapters ago you had never seen a vector. Now you can hand-walk attention, read a loss curve, and decode "a rank-16 LoRA, 8K context, Q4" as if it were an ordinary sentence. But let's step back and ask the question this whole site pointed at without ever answering head-on: what kind of thing is an LLM? Is it "artificial intelligence"? "Machine learning"? "Deep learning"? You hear all three thrown at it — and this is the chapter where they get sorted.

Fair warning: this is the one chapter that builds no new machine. It zooms all the way out, draws the family map of the field, drops a pin on exactly where the LLM sits, and shows you that the toolkit in your hands was never an "LLM toolkit" — it was the field's shared toolkit all along.

I won't keep you waiting for the answer, because this site never teases. An LLM is all three of those words at once — because they are nested rings, not rival camps, and the LLM lives in the innermost one. The rest of the chapter earns that sentence.

Everything was one idea in disguise

Let me recompress the entire site into a single sentence: an LLM is a gigantic function that takes a context of tokens and returns a probability for every possible next token — and "training" was nothing but nudging that function's numbers, one small downhill step at a time, until its guesses stopped being wrong so often. Here is the job itself, from Chapter 9:

In words: given the run of tokens so far, hand back the probability of every token that could come next. That distribution is the LLM's entire output.

Walk your own bench and watch the parts line up: tokens turned text into numbers (Ch. 7); embeddings gave every token a direction (Ch. 8); attention and the transformer are the shape of the function (Ch. 10–11); softmax turned raw scores into that probability distribution (Ch. 9); cross-entropy loss graded each guess (Ch. 12); gradients pointed downhill and gradient descent took the steps (Ch. 13–14); and the training run repeated that step trillions of times (Ch. 15).

That frame has a name. It is function-fitting, and it is the whole definition of machine learning: you don't tell the machine the rules; you give it a flexible function with millions of adjustable numbers and a way to measure wrongness, and you let a downhill walk find the numbers. An LLM is a spectacularly large instance of that recipe — not a different kind of thing. Strip away the tokens and the trillions of steps, and the machinery of learning fits on one line:

In words: take every number in the model — $\theta$ (theta, all the parameters) — and nudge it a small step, of size $\eta$ (eta, the learning rate), in the direction that most reduces the loss. That direction is $\nabla L$ (the gradient of the loss, the compass we walk against). Then do it again.

That one line trains an LLM, a spam filter, a self-driving car's vision system, and a sonar classifier. The same line — only the function and the data change.

Four nested rings

Now let's sort those three words, widest to narrowest, as rings inside rings. Picture first, definitions after:

Ring 1, the widest, is artificial intelligence (AI): the oldest, broadest goal — make a machine do something that looks intelligent. Crucially, this ring includes methods with no learning at all: 1990s chess programs, tax software applying rules an accountant wrote, a thermostat's logic. Someone wrote those rules by hand. That is AI, and it is not machine learning.

Ring 2, inside it, is machine learning (ML): the subset where nobody writes the rules — the machine finds them by fitting a function to data. The moment a system learns from examples instead of following hand-written instructions, it steps from the outer ring into this one.

Ring 3, inside that, is deep learning (DL): the flavor of ML where the flexible function is a neural network — many layers of the weighted-sum-then-squash idea, stacked so early layers learn simple features and later ones build on them. Not all ML is deep: a line-fitter learns, but it is not a deep network. Deep learning is what made vision, speech, and language suddenly work in the 2010s.

Ring 4, the innermost, is where your pin goes: the transformer (Chapter 11) and the LLM built from it — one deep-learning architecture, specialized for sequences of tokens, trained on a mountain of text. So the pin sits in all four rings at once, because they are nested: an LLM is a transformer, which is deep learning, which is machine learning, which is AI. Every "is it X?" answers yes; it is only a question of how far in you zoom.

One honest widening, so you don't over-fit to the center: the transformer is not the only tenant of the deep ring, and the deep ring is not all of ML. Beside the transformer live other architectures (next section), and outside the deep ring sit whole families — decision trees, forests, clustering — that never touch a neural network and still quietly win on huge classes of real problems. The center is famous; it is not the whole map.

Three ways a machine can learn

Zoom into the machine-learning ring and ask a different question: how does a model learn — what does it learn from? There are three classic answers, and the whole field organizes around them.

Way one is supervised learning: examples come with the right answers — photos labeled cat/dog, emails labeled spam/not, houses labeled with their price. The model learns to predict the answer from the input, and the loss measures how far its guess sits from the true label (Chapter 12's whole idea). This is the workhorse: classification and regression.

Way two is unsupervised learning: no answers at all, just raw data, and the job is to find structure in it — group customers into natural clusters, compress a thousand features into the two that matter, spot the odd one out. Nobody labeled anything; the model organizes the pile.

Way three is reinforcement learning (RL): no fixed dataset — an agent acts in a world, gets a reward or a penalty, and adjusts to earn more reward over time. This is how machines learned to play Go and steer robots, and it is deliberately harder, because the right answer is never handed over — it has to be discovered through consequences.

Now the beat that makes the whole site click: where did the LLM learn? It used two of the three, in sequence. Pretraining (Chapter 15) is a sly form of supervised learning called self-supervised learning: the "label" for each position is just the next token, already sitting there in the text — so the internet becomes a labeled dataset with no human labeling at all, which is exactly why LLMs could train on trillions of tokens. Then the assistant polish (Chapter 16) is a reinforcement-learning-style finish that rewards the responses humans prefer — RLHF, or its leaner cousin DPO. The chatbot you talk to is a self-supervised base model with a preference-tuned finish.

Data has a shape — match the tool to it

The last big idea of the map, and the sister site's whole third act in one sentence: the transformer is not the universal answer. Different data has different shape, and the winning architecture is usually the one whose structure matches the data's.

Text and audio are sequences — order matters, and each piece looks at the others — so the transformer fits (and, before it, the recurrent networks it dethroned). The sister site's Remembering Machines tells the honest story of why attention replaced recurrence.

Images are grids — a pixel's meaning depends on its neighbors, and the same little pattern (an edge, an eye) can appear anywhere — so the winning tool is the convolutional neural network (CNN), which slides small learned filters across the grid and reuses the same weights everywhere: Seeing Machines.

Some data is a graph — nodes joined by edges, with no natural order: social networks, road maps, and communications networks, where the stations are nodes and the links between them are edges. The tool is the graph neural network (GNN), which learns by passing messages between connected nodes. This is how machine learning predicts a node's role from the structure of the network around it — the sister site's Networks of Relationships.

And a great deal of the world's most valuable data is just a table — rows and columns, like a spreadsheet of customers or lab measurements — where, surprisingly often, no neural network wins at all. The champions here are the classical methods: decision trees, random forests, gradient boosting. On tabular data they routinely beat deep networks while training in seconds and staying interpretable (The Classical Toolbox). The honest headline: deep learning is the right answer for perceptual, high-dimensional data — pixels, audio, language — and often the wrong one for a spreadsheet. Real expertise is reading a problem's data shape and choosing the right family: a forest for a spreadsheet, a GNN for a network, a transformer for a sequence.

Your toolkit already travels

Here is the chapter's gift: you did not learn a pile of "LLM facts." You learned the field's shared vocabulary, and it transfers to every architecture on that map with almost nothing to relearn.

Vectors and embeddings (Ch. 1, 8): every model represents its inputs as vectors — a CNN turns an image patch into a vector, a GNN gives every node a vector, a recommender gives every song a vector. "Turn the thing into a vector" is step one of all of machine learning.

The dot product (Ch. 2), the operation I promised was the most important on the site, measures similarity everywhere: attention scores are dot products, an SVM's kernel is a dot product, a GNN's messages are weighted by dot products, a recommender ranks songs by the dot product of their vectors. Learn it once, meet it forever.

Matrices (Ch. 3), gradient descent and backprop (Ch. 13–14), the loss (Ch. 12), softmax (Ch. 9): every neural network is stacked matrix transformations, trained by gradient descent walking downhill on a loss with backprop assigning the blame, and nearly every classifier ends in a softmax turning scores into a probability distribution. A CNN and a GNN and your transformer are, under the hood, the same three verbs: represent as vectors, transform with matrices, train downhill.

So carry this as a rule: when you open the sister site and meet a convolution or a message-passing step for the first time, it will feel new for about a paragraph — and then you'll see it's your old friends in a new arrangement. You are not starting over; you're reading the same language with a new accent.

Where you're headed

This site answered one deep question — how does an LLM actually work — by building it from the first vector up. The sister site, AI/ML from the Ground Up, asks the broader one: how does the whole field work, and how do you choose the right model for any problem? It rebuilds machine learning from first principles — line-fitting, the neuron, the classical toolbox, CNNs, RNNs, GNNs, generative models, reinforcement learning — and ends by teaching you to pick the right tool given a problem's shape.

Your footing there is unusually good, because you arrive already fluent in the math. The sister site opens with a condensed Math Toolkit you can skim as a victory lap or skip outright — every idea in it, you built in depth here. From there, jump to whatever pulls you: the classical methods that win on tables, how machines see, how they learn on graphs and networks, or the capstone that ties it together.

There is a third site, and for many readers it is the better next step — especially if you arrived here from AI/ML from the Ground Up and have already walked its floors. How AI Makes Images takes everything you built here — vectors, the dot product, gradient descent, backprop, attention — and points it at pictures instead of words. It answers the question people ask second, right after the one about ChatGPT: when a machine makes a picture out of nothing, what is actually happening? The answer runs on your machinery. A network predicts noise instead of the next token, graded by a loss and trained downhill exactly as here; the prompt steers the image through the same attention you can now compute by hand. It is the third rung, and it assumes no more than you already have.

And to close the way we opened: you asked how these things actually work, and now you know — not by analogy, but by the arithmetic. You can check attention by hand, read a loss curve, and place any model you meet on the family map. That was the promise. The door to the rest of the field is open, and you already speak the language. Go build something.