You’re inspecting a model export and a familiar word stops being clear. Was that shape counting samples or features? Was the checkpoint the best score or a saved state? This reference sends every term back to its first explanation, where the diagrams and worked values give the definition something to attach to.
Search by a term or an alias, then follow its link to the chapter that first explained it. Each entry uses the same wording as the definition that pops up in the chapters. When a later chapter sharpens a term, that refinement appears with its own link.
A glossary entry points back to where the idea was taught.
A short definition recovers a word; the source chapter recovers the operation. For “gradient,” go back to the individual slopes and the weight update. For “shape,” go back to the array and its axes. A term pays off when you can point at the object in a real computation, give its units or dimensions, and say which operation eats it next.
The symbol stands for an object
A symbol names an object or an operation, and its typography says which. Bold lowercase letters are vectors, bold uppercase letters are matrices, and plain subscripts name individual entries. The distinction earns its keep when an export holds many arrays: one weight value and the matrix around it belong to different descriptions of the same computation. Neither notation changes the numbers in the file.
Notation identifies the object and the address of its entries.
Read each symbol in the table below with its local context. Theta can name all model parameters in a training objective, and an angle in an earlier geometry example used theta with that meaning stated on the spot. In training, eta is the learning rate and sigma is sigmoid. We spell tanh and ReLU out, so the activations stay distinguishable.
Object
Notation
Read aloud
scalar
italic lowercase
"a number x"
vector
bold lowercase
"the vector v"
vector components
subscripts
"v-one, v-two"
column vector literal
bmatrix
"the vector two-one"
matrix
bold uppercase
"the matrix W"
matrix entry
W_{ij}
"the entry in row i, column j"
dot product
center dot
"v dot u" — mint vector FIRST, see rule 8
vector length / norm
\|\mathbf{v}\|
"the length of v"
angle between vectors
\theta
"theta"
cosine similarity
\cos\theta
"the cosine of the angle"
function
f(x)
"f of x"
exponential
e^x
"e to the x"
natural log
\ln x
"the natural log of x"
derivative
\frac{df}{dx}
"the derivative of f with respect to x"
partial derivative
\frac{\partial L}{\partial w}
"the partial derivative of the loss with respect to the weight w"
gradient
\nabla L
"the gradient of the loss"
sum
\sum_{i=1}^{n}
"the sum over i from one to n"
probability
p(x)
"the probability of x"
conditional prob.
p(\text{next} \mid \text{context})
"probability of the next token given the context"
loss
L or L(\theta)
"the loss"
model parameters (all of them)
\theta
"theta — the parameters"
a single weight
w
"a weight"
learning rate
\eta
"eta, the learning rate"
sigmoid (LLM/AIML only — banned in `diffusion/`, see Cross-site collisions)
temperature (LLM/AIML only — see Cross-site collisions)
T
"temperature"
embedding dimension
d (LLM site: d_{\text{model}} once introduced)
"d, the number of dimensions"
vocabulary size
V
"V, the vocabulary size"
context length
n tokens
attention matrices
\mathbf{Q}, \mathbf{K}, \mathbf{V}_{\!a} — NOTE: attention value matrix written \mathbf{V}_{\!a} ONLY if vocabulary V is in scope on the page; otherwise plain \mathbf{V} with a sentence noting it is unrelated to vocabulary size
\{(x^{(i)}, y^{(i)})\}_{i=1}^{N} — introduce gently; prefer "our N examples" in prose
training mean (training/data-into-tensors only)
m
"m, the mean computed from training rows"
training standard deviation (training/data-into-tensors only)
s
"s, the standard deviation computed from training rows"
prediction
\hat{y}
"y-hat, the model's guess"
A matrix and one of its entries
Let W be the identity matrix [[1,0],[0,1]] and x be the vector (1,2). The first output is 1×1+0×2=1. The second is 0×1+1×2=2. Thus this matrix leaves the input unchanged. The mathematical entry W₁₂ is the first row, second column, which contains zero. In Python or JSON the same entry is [0][1], because code indexing starts at zero rather than one.
In words: this identity matrix leaves the input vector unchanged.
Return to containers and indices for the address convention, or space that stretches for the matrix’s action on a point. A two-by-two identity is also a useful first check of a new calculation: if a supposed identity transformation changes (1,2), inspect the axis order and multiplication rule before trusting a more complicated example.
Find the idea in the workflow
The workflow map starts with containers, because every later stage takes in and hands back shaped numbers. A transform changes those values. A forward pass produces predictions, a loss grades them, a gradient gives local sensitivity, and an update changes the parameters. The dashboard shows measurements from that process. Architecture choices rearrange the operations; data and evaluation choices decide what the measurements mean.
The definitions connect back to one continuous training workflow.
Each node returns to the chapter that owns that stage. The arrows show prerequisite direction, not an order in which software executes. A data split gets chosen before training, for instance, even though the book explains training first with fixed samples. The map is a reading route through ideas; the training-loop diagram is a route through a computation.
Use that experiment to separate three questions. What are the values? How are they arranged? Which axis means what? Two arrays can hold equal counts with different shapes, and two equal shapes can give their axes different meanings. So a transpose, a flattening, and a changed feature order are not interchangeable descriptions. The source chapters say when each operation fits and how to check it numerically.
The glossary can’t decide whether your model is useful. It can help you state the evidence precisely enough to decide. Name the task, the baseline, the split, the metric, and the acceptance criterion. Say whether a displayed loss is in-sample, validation, or final test. Explain what one exported weight multiplies and which preprocessing makes that input mean something.
The reference also helps when you explain a result to someone else. Start with the input and the question the model was trained to answer. Show a prediction and the rule that scored it. Then show the update mechanism and the independent evaluation evidence. Your listener needn’t memorize every symbol to follow the chain, as long as each object is defined where it first matters. The glossary is the return path when that chain needs a closer look.
What you now know
Definitions point to their first explanation and worked example.
Notation distinguishes arrays from their entries.
The same numeric objects reappear throughout training.
Where we’re headed
Go back to your model export with this reference beside it, or on to the language-model book to follow the same operations at larger scale. Continue the story.