Part IV · Ch. 25 — Glossary and Notation

Part IV · Chapter 25 of 29

Glossary and Notation

Find a term, its meaning, and the chapter that introduced it


Return to the first definition

You’re inspecting a model export and a familiar word stops being clear. Was that shape counting samples or features? Was the checkpoint the best score or a saved state? This reference sends every term back to its first explanation, where the diagrams and worked values give the definition something to attach to.

Search by a term or an alias, then follow its link to the chapter that first explained it. Each entry uses the same wording as the definition that pops up in the chapters. When a later chapter sharpens a term, that refinement appears with its own link.

The gradient tooltip connects to its first explanation and worked example.
A glossary entry points back to where the idea was taught.

A short definition recovers a word; the source chapter recovers the operation. For “gradient,” go back to the individual slopes and the weight update. For “shape,” go back to the array and its axes. A term pays off when you can point at the object in a real computation, give its units or dimensions, and say which operation eats it next.

The symbol stands for an object

A symbol names an object or an operation, and its typography says which. Bold lowercase letters are vectors, bold uppercase letters are matrices, and plain subscripts name individual entries. The distinction earns its keep when an export holds many arrays: one weight value and the matrix around it belong to different descriptions of the same computation. Neither notation changes the numbers in the file.

Scalar, vector, identity matrix and entry W₁₂; the same entry has JSON address [0][1].
Notation identifies the object and the address of its entries.

Read each symbol in the table below with its local context. Theta can name all model parameters in a training objective, and an angle in an earlier geometry example used theta with that meaning stated on the spot. In training, eta is the learning rate and sigma is sigmoid. We spell tanh and ReLU out, so the activations stay distinguishable.

ObjectNotationRead aloud
scalaritalic lowercase"a number x"
vectorbold lowercase"the vector v"
vector componentssubscripts"v-one, v-two"
column vector literalbmatrix"the vector two-one"
matrixbold uppercase"the matrix W"
matrix entryW_{ij}"the entry in row i, column j"
dot productcenter dot"v dot u" — mint vector FIRST, see rule 8
vector length / norm\|\mathbf{v}\|"the length of v"
angle between vectors\theta"theta"
cosine similarity\cos\theta"the cosine of the angle"
functionf(x)"f of x"
exponentiale^x"e to the x"
natural log\ln x"the natural log of x"
derivative\frac{df}{dx}"the derivative of f with respect to x"
partial derivative\frac{\partial L}{\partial w}"the partial derivative of the loss with respect to the weight w"
gradient\nabla L"the gradient of the loss"
sum\sum_{i=1}^{n}"the sum over i from one to n"
probabilityp(x)"the probability of x"
conditional prob.p(\text{next} \mid \text{context})"probability of the next token given the context"
lossL or L(\theta)"the loss"
model parameters (all of them)\theta"theta — the parameters"
a single weightw"a weight"
learning rate\eta"eta, the learning rate"
sigmoid (LLM/AIML only — banned in `diffusion/`, see Cross-site collisions)\sigma(z) = \frac{1}{1 + e^{-z}}"sigma of z"
softmax\mathrm{softmax}(\mathbf{z})_i = \frac{e^{z_i}}{\sum_j e^{z_j}}"softmax"
temperature (LLM/AIML only — see Cross-site collisions)T"temperature"
embedding dimensiond (LLM site: d_{\text{model}} once introduced)"d, the number of dimensions"
vocabulary sizeV"V, the vocabulary size"
context lengthn tokens
attention matrices\mathbf{Q}, \mathbf{K}, \mathbf{V}_{\!a} — NOTE: attention value matrix written \mathbf{V}_{\!a} ONLY if vocabulary V is in scope on the page; otherwise plain \mathbf{V} with a sentence noting it is unrelated to vocabulary size"queries, keys, values"
attention formula\mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V}) = \mathrm{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^\top}{\sqrt{d_k}}\right)\mathbf{V}
LoRA update\mathbf{W}' = \mathbf{W} + \frac{\alpha}{r}\,\mathbf{B}\mathbf{A} (rank r)
dataset\{(x^{(i)}, y^{(i)})\}_{i=1}^{N} — introduce gently; prefer "our N examples" in prose
training mean (training/data-into-tensors only)m"m, the mean computed from training rows"
training standard deviation (training/data-into-tensors only)s"s, the standard deviation computed from training rows"
prediction\hat{y}"y-hat, the model's guess"

A matrix and one of its entries

Let W be the identity matrix [[1,0],[0,1]] and x be the vector (1,2). The first output is 1×1+0×2=1. The second is 0×1+1×2=2. Thus this matrix leaves the input unchanged. The mathematical entry W₁₂ is the first row, second column, which contains zero. In Python or JSON the same entry is [0][1], because code indexing starts at zero rather than one.

$$\mathbf{W}\mathbf{x}=\begin{bmatrix}1&0\\0&1\end{bmatrix}\begin{bmatrix}1\\2\end{bmatrix}=\begin{bmatrix}1\\2\end{bmatrix}$$

In words: this identity matrix leaves the input vector unchanged.

Return to containers and indices for the address convention, or space that stretches for the matrix’s action on a point. A two-by-two identity is also a useful first check of a new calculation: if a supposed identity transformation changes (1,2), inspect the axis order and multiplication rule before trusting a more complicated example.

Find the idea in the workflow

The workflow map starts with containers, because every later stage takes in and hands back shaped numbers. A transform changes those values. A forward pass produces predictions, a loss grades them, a gradient gives local sensitivity, and an update changes the parameters. The dashboard shows measurements from that process. Architecture choices rearrange the operations; data and evaluation choices decide what the measurements mean.

learning map1Containers2Transforms7Forward8Loss9Gradient11Update12Dashboard13Architectures19Data21Evaluation23Own modelArrows show which chapters rely on which; select a box to open its chapter.
The definitions connect back to one continuous training workflow.

Each node returns to the chapter that owns that stage. The arrows show prerequisite direction, not an order in which software executes. A data split gets chosen before training, for instance, even though the book explains training first with fixed samples. The map is a reading route through ideas; the training-loop diagram is a route through a computation.

Use that experiment to separate three questions. What are the values? How are they arranged? Which axis means what? Two arrays can hold equal counts with different shapes, and two equal shapes can give their axes different meanings. So a transpose, a flattening, and a changed feature order are not interchangeable descriptions. The source chapters say when each operation fits and how to check it numerically.

Where this shows up when you train

A term search should bring back a usable example as well as a short definition. Go to loss and its worked probability, the seventeen-entry backward calculation, or the difference between best score and saved state. For the final artifact, go to the sample table’s model.json export and match a stored row to its preprocessed inputs.

The glossary can’t decide whether your model is useful. It can help you state the evidence precisely enough to decide. Name the task, the baseline, the split, the metric, and the acceptance criterion. Say whether a displayed loss is in-sample, validation, or final test. Explain what one exported weight multiplies and which preprocessing makes that input mean something.

The reference also helps when you explain a result to someone else. Start with the input and the question the model was trained to answer. Show a prediction and the rule that scored it. Then show the update mechanism and the independent evaluation evidence. Your listener needn’t memorize every symbol to follow the chain, as long as each object is defined where it first matters. The glossary is the return path when that chain needs a closer look.

What you now know

  • Definitions point to their first explanation and worked example.
  • Notation distinguishes arrays from their entries.
  • The same numeric objects reappear throughout training.

Where we’re headed

Go back to your model export with this reference beside it, or on to the language-model book to follow the same operations at larger scale. Continue the story.