What did we build?
We built a large language model. Given tokens so far, it outputs a distribution over possible next tokens:
$$p(\text{next token} \mid \text{context})$$In words: given the run of tokens so far, hand back the probability of every token that could come next. That distribution is the LLM's entire output.
Sampling picks a continuation. Training adjusts the function that produces those probabilities. Everything from embeddings to attention helps shape that function.
The same idea in every chapter
Function-fitting connects our chapters: choose a flexible function, measure its errors with a loss, then improve its parameters. For a differentiable model, our update was:
$$\theta \leftarrow \theta - \eta\,\nabla L$$In words: take every number in the model, written $\theta$ (theta, all the parameters), and nudge it a small step of size $\eta$ (eta, the learning rate) in the direction that most reduces the loss. That direction is $\nabla L$, the gradient of the loss, and we walk against it. Then do it again.
Worked example: the same update, a smaller model
Let a line predict $y=wx$, with $w=2$ and loss derivative 4. At learning rate 0.1, gradient descent gives $w_{\text{new}}=2-0.1\times4=1.6$. The transformer updates many parameters by the same rule.
Other learning methods fit functions without gradients. A decision tree, for example, selects splits in the data. We should carry the general idea without insisting that every model uses our optimizer.
Four nested rings
Artificial intelligence (AI) includes hand-written rules and learned systems. Machine learning (ML) is the learning part. Deep learning (DL) uses layered neural networks. Our LLM belongs inside all three; the transformer is its architecture.
Three ways a machine can learn
| Approach | Signal | Example |
|---|---|---|
| Supervised learning | Targets paired with inputs | Classification: spam or legitimate mail; regression: house price |
| Unsupervised learning | Structure in unlabeled data | Cluster customers or compress measurements |
| Reinforcement learning (RL) | Rewards for actions | Game play or robot control |
Self-supervised learning supplies its own targets. In language pretraining, the next token is already in the text. This provides a supervised prediction objective without someone labeling every position. Terminology sometimes places self-supervision within the broader unsupervised family.
Worked example: find the free labels
For “the cat sat,” two training pairs are “the” → “cat” and “the cat” → “sat.” The text supplies both answers. At inference we no longer have the next answer; we predict it.
Assistant training can add supervised demonstrations, then preferences. RLHF uses a reward signal; DPO fits preferences directly without a reinforcement-learning rollout loop. These are different ways to improve behavior, not mandatory stages for every LLM.
Match the tool to the shape of the data
| Data | Candidate family | Useful structure |
|---|---|---|
| Sequences: text, audio | Transformers; recurrent networks | Order and relationships across positions |
| Grids: images | Convolutional neural network (CNN); vision transformer | Repeated local patterns |
| Graphs: roads, social or communications links | Graph neural network (GNN) | Neighbors joined by edges |
| Tables: customers, measurements | Classical methods | Features in columns; trees are useful baselines |
| Data shape | Candidate family | Sister-book chapter |
|---|---|---|
| Sequence: text, audio; order matters | Transformer | Remembering Machines |
| Grid: images; neighbors matter | CNN | Seeing Machines |
| Graph: linked entities; connections matter | GNN | Networks of Relationships; link analysis |
| Table: rows and columns | Trees, forests, boosting; often strong here | The Classical Toolbox |
These are starting hypotheses. Dataset size, accuracy, interpretability, latency, and cost decide the useful model. CNNs and transformers can both process images; deep networks and trees can both process tables.
Your toolkit already travels
| Tool | Other jobs |
|---|---|
| Vectors and embeddings | Represent image patches, graph nodes, or recommended items |
| Dot products | Linear scores, similarity, and some attention or kernel methods |
| Matrices | Learned transformations in many neural networks |
| Loss, gradients, backpropagation | Train differentiable models for images, audio, or control |
| Softmax | Turn competing category scores into probabilities |
See it move
Why the LLM cares
An LLM is one member of a larger field. We can inspect its objective, architecture, data, and deployment costs using the same questions we ask of other models. Knowing its arithmetic helps us choose when it fits the problem.
Where you're headed
AI/ML from the Ground Up widens the view: line-fitting, neurons, classical methods, CNNs, RNNs, GNNs, generative models, and reinforcement learning. Review its Math Toolkit as needed, then explore trees and forests, vision, sequences, graphs, or choosing a tool.
If we have already walked that route, How AI Makes Images points the same math at pictures. Its model learns to predict noise, and attention connects image generation to a text prompt. We arrive with useful tools.
What you now know
- Our LLM belongs to AI, machine learning, and deep learning.
- Training fits a function to an objective.
- Learning signals include labels, data-derived targets, structure, and rewards.
- Data shape suggests model families; evaluation chooses between them.
- Vectors, matrices, losses, and gradients travel beyond language models.
Keep going
We can now compute attention by hand, read a loss curve, and explain where an answer comes from. Choose a question on either sister site and keep building.