Choosing Your Tool

The skill above the tools

Twelve chapters ago, a "model" was a mystery. Now you can fit a line, grow a forest, stack a deep network, slide a convolution across an image, pass messages on a graph, denoise an image into existence, and train an agent by reward. You own the tools. But owning tools is not the job. A carpenter is not defined by having a saw and a chisel — they are defined by knowing, at a glance, which one this cut needs. This final chapter is about that judgment: given a problem you have never seen, how do you choose?

Here is the plan: a compact decision framework of five questions, spent on six real scenarios walked end to end — finishing on one where the right answer is a graph neural network and every other tool is quietly wrong — then an interactive chooser and the hard-won principles that outlast any specific model.

The whole lesson fits in one contrast. A beginner asks, "what's the best model?" — and there is no answer, because "best" has no meaning without a problem attached. A practitioner asks, "what does this problem look like?" and lets the answer choose the model. The rest of this is how to look.

The five questions

Here is the framework: five questions, in priority order. The two at the top do most of the work; the last three break ties. The whole map fits on one page — keep it in view.

Question 1 — what shape is the data? This is the most important question, because the winning architecture is usually the one whose structure matches the data's — the thesis the whole third part of this site defended. Four shapes: a table (rows and columns, a spreadsheet); a grid (an image, where neighboring pixels matter); a sequence (text or audio, where order matters); or a graph (nodes joined by edges — contact networks, social networks, road maps — with no natural order). Match the shape and you are most of the way home: table → classical methods, grid → CNN, sequence → transformer, graph → GNN.

Question 2 — what can you learn from? The three ways to learn, as a triage. Do you have labels — examples with the right answer attached? Then it is supervised, classification or regression. No labels, just raw data? Then unsupervised — k-means or PCA to find structure. No dataset at all, only a reward from acting in a world? Then reinforcement learning. This one question rules out most of the toolbox.

The last three are tie-breakers, one line each. Question 3 — how much data? Deep networks are hungry: thousands to millions of examples. With only hundreds, a classical method or a pre-trained model is safer. Question 4 — do you need to explain each decision? In medicine, credit, and law, a model that can show its reasoning — a tree, a linear model — can beat a more accurate black box you are not allowed to deploy. Question 5 — what is your budget in compute, time, and expertise? A boosted tree trains in seconds on a laptop; a large network can cost real money and a team.

That points at the honest default running through this whole site: reach for the simplest tool that fits the shape, and make a deep network earn its place. Deep learning is the right answer for perceptual, high-dimensional, structured data — pixels, audio, language, graphs — and often the wrong answer for a spreadsheet. Now let us watch the five questions decide.

Six problems, six choices

Six real problems, each run through the five questions out loud. Five are here; the sixth gets its own section because it is the punchline. Follow the scorecard row by row.

Scenario 1 — predict a house's price from a spreadsheet of features: square footage, bedrooms, zip code. Shape: a table. Learn from: labels — each house has a sold price, a number, so this is regression. Size: a few thousand rows. Verdict: start with plain linear regression as a baseline — honest and instant — then reach for gradient-boosted trees from the classical toolbox for accuracy. A neural network here would train slower, overfit easier, and explain less: the wrong tool for a spreadsheet.

Scenario 2 — split your customers into natural groups you never defined in advance. Shape: a table again. Learn from: no labels at all — nobody tagged these customers. Verdict: this is unsupervised, so k-means to find the clusters and PCA to squash the features down and see them, both from the classical toolbox. No neural network, no labels — just find the structure that is already there.

Scenario 3 — flag likely pneumonia on chest X-rays. Shape: a grid of pixels, where a pixel's meaning is set by its neighbors. Learn from: labels — radiologist-confirmed cases. Size: large. Verdict: a convolutional neural network from seeing machines, the tool built for grids, likely starting from one pre-trained on millions of images and fine-tuned on the X-rays. Note the honest tension: medicine wants explanations, and a CNN is less transparent than a tree — so in practice it is paired with heat-maps showing where it looked, with a doctor in the loop. The shape forces a deep network; the stakes force care around it.

Scenario 4 — build a support chatbot that answers in fluent prose. Shape: a sequence — language, where order and long-range context carry the meaning. Learn from: a mountain of unlabelled text (self-supervised pretraining) plus human preferences for the polish. Verdict: a transformer — a large language model, the subject of the entire sister site — finished with the RLHF from learning by doing so it is helpful and safe, not just fluent. The shape (a sequence) and the task (generation) both point at one family.

Scenario 5 — teach a warehouse robot to grasp and stack items nobody demonstrated. Shape: a stream of decisions over time. Learn from: no labeled "correct grasp" exists — only a reward when the item ends up stacked. Verdict: reinforcement learning from learning by doing, trained in a physics simulator where it can fail ten thousand times for free before touching a real shelf. No labels, a reward-per-outcome — the fingerprint of RL.

The one where the answer is a GNN

Now the finale, and it is a real one. Scenario 6 — decide whether a cluster of accounts is a coordinated network, one organization running a controlled net, from its contact pattern alone, before anyone reads a word of the traffic. A real, high-value triage task — and the one where every tool you would instinctively reach for is subtly wrong except one.

Run the five questions, but slow down on the shape, because the shape is the lesson. Learn from: labels — tens of thousands of past clusters whose character was eventually established either way, so supervised classification. The hard question is the shape. A contact pattern is not a table, not an image, and not really a sequence — it is a graph: the accounts are the nodes, the contacts between them are the edges, and there is no natural left-to-right order. It is the same network no matter which account you decide to "start" from.

Here is the move that earns the chapter — watch each obvious tool throw away the signal. Flatten the network into a table of counts (how many accounts, how many contacts, average messages per day) and feed it to a boosted tree: fast, but you have deleted the structure — a tightly controlled hub-and-spoke net and a loose social crowd can post identical counts while being wired completely differently, and the table cannot tell them apart. Treat it as an image: a network is not a grid, and a picture of one imposes a fake 2-D layout that the drawing software chose, not the data. Write it as a sequence — the raw contact log, every event in time order — and hand it to a transformer: this genuinely works, and people do it. But the log is an arbitrary linearization of something that was never a line, so the model spends its capacity re-learning the connection structure you already knew and threw away.

Now the fit. A graph neural network from networks of relationships takes the network as a graph and never flattens it. Each account starts with a vector describing itself — activity volume, timing pattern, how long it has been live — then repeatedly passes messages to the accounts it contacts and updates. After a few rounds, every account's vector reflects its whole neighborhood, a readout pools them, and the network reads the verdict off the assembled picture. The structure every other tool discarded is exactly the structure the GNN computes on. That is "the shape of the data chose the model," made concrete: the tool whose wiring matches the network's wins.

And this is not just about comms. The same fingerprint — entities connected by relationships, no natural order, where the connections carry the meaning — points at a GNN every time: fraud rings in a transaction network, recommendations across a social graph, traffic over a road map, a molecule's properties read from its atoms and bonds. When you catch yourself saying "it is really about how things are connected," you have found a graph, and you have found your tool.

Walk it yourself

The five questions and the six scenarios were the map; now you drive it. Answer the same questions about a problem — its data shape, what you can learn from, how much data, how much you must explain — and watch the field guide light a path to a recommended family, with a one-line reason and a link to the chapter that teaches it.

One honest caveat, said once: this is a guide to the first tool to try, not a verdict. Real projects start with the simplest model that fits, measure it, and only climb to something heavier if the numbers demand it. The framework points you at the right neighborhood; the data has the final word.

The judgment that outlasts the tools

A few durable principles — the things still true when today's models are museum pieces.

Principle one — start simple. A baseline you understand, built fast, beats a sophisticated model you cannot debug, and it tells you whether the hard problem is even hard. Reach for the simplest tool that fits, and make every step up in complexity earn itself against the baseline's number.

Principle two — no free lunch. There is no single best model, only a best fit — the reason this chapter exists. A method that dominates on images can lose to a decade-old algorithm on a spreadsheet. "What is the state of the art?" is the wrong question; "what fits this?" is the right one.

Principle three — structure is signal. The biggest wins in this whole site came from wiring the model to match the data's shape: convolution for grids, attention for sequences, message-passing for graphs. When a general tool underperforms, ask what structure in your data it is ignoring. The right inductive bias beats brute force more often than the headlines suggest.

Principle four — the cost of the wrong tool is real. Aiming a deep network at a small tabular problem is one of the most common and expensive mistakes in the field — slower, less accurate, unexplainable, and harder to ship, all at once. Choosing well is the difference between a project that works and one that quietly does not.

And now the deepest thread: under all the choosing, there is one engine. Whatever family you pick, if it learns, it almost certainly learns the same way — a flexible function, a loss that measures wrongness, and gradient descent nudging the numbers downhill. Here it is, one last time:

In words: nudge every parameter — $\theta$, theta, the whole bag of numbers the model learns — a small step of size $\eta$ (eta, the learning rate) against $\nabla L$, the gradient of the loss, the direction in which wrongness climbs fastest. Step against it and wrongness falls. The tools differ in their shape; the learning underneath is one idea, held since fitting a line. Choosing well is choosing the right shape to wrap around that one engine.

Where you've been — and the door across the hall

You started this site unable to read a vector. You can now stand in front of an unfamiliar problem, read the shape of its data, name the family that fits, and say why — including the unglamorous truth that a boosted tree often beats a neural network, and the payoff that a web of relationships wants a graph network. That was the whole promise: not a pile of model facts, but the judgment to choose.

So let me point across the hall, because this is a two-way door. The sister site, How LLMs Actually Work, takes the single most important model in the field — the transformer you would pick for Scenario 4 — and builds it from the first vector to a working assistant, in full arithmetic detail. Because you already own the math, you can skim its toolkit chapter and go straight to how text becomes numbers, how attention works, and how a next-token predictor becomes the chatbot you use. Its own final chapter zooms back out to this exact question of choosing, from the other side — the two sites meet in the middle.

To close the way we opened: you asked how these things actually work — the whole family of them, not just the famous one — and now you can answer with arithmetic instead of awe, and choose with judgment instead of hype. The toolbox is yours, the judgment is yours, and the door across the hall is open. Go build the right thing.