AI/ML Q&A

AI/ML Q&A

This is a Q&A I maintain. Colleagues and customers ask questions about AI and machine learning, and some turn out to be central: the kind where a real answer moves someone from hearing about these tools to using them. I answer those here, properly: picture first, then math you can check by hand, then a video for the couch. New questions get added as they come in.

Are all AIs just predictive number generators?

Asked by MAJ Taylor Wulff-Morrison, USMC

Where the randomness lives, how to switch it off, and which AIs are not this at all.

16 min
What do LLMs have to do with entropy, thermodynamics and information theory?

Asked by AG1 Derek Conklin, USN

Learning is compression, sampling is thermodynamics, and entropy is the exchange rate.

20 min
What really are embeddings?

Asked by LT David Robinson, USN

Exact at the address, approximate at the angle, and a good deal more than search.

16 min
Why are the nonlinear operations the magic?

Asked by Andrew Reynolds, Lincoln, Nebraska

Why stacking linear layers buys nothing, what one fold buys, and a complete learning step by hand.

16 min
Why are there so many tokens in a vocabulary?

Asked by Matthew Reid, Plano, Texas

Why the count is a chosen budget of reusable pieces, and why English-only would not make it small.

13 min
How does next-token training turn into reasoning?

Asked by LT David Robinson, USN

Why the objective is not the mechanism, where reasoning happens, and what parameters buy.

20 min
How does the model give a probability to a phrase, or a meaning?

Asked by MAJ Taylor Wulff-Morrison, USMC (follow-up to question 1)

Multiply the faces you took; “which meaning” is where the mass lands, and context moves it.

18 min