Part IV · Ch. 24 — Train Your Own Model

Part IV · Chapter 24 of 29

Train Your Own Model

Run trainkit on your data and read your weights


Open the shipped package

Now train a small model and explain what the resulting file holds. Start with a supplied sample, so the path, the shapes, and the output can be checked before the data changes. Then swap in your own examples and keep the same evidence: command, split, metric, baseline, and one weight row whose multiplication you can explain.

A command line is a text interface for running a program with arguments. Download trainkit.zip and extract it. Open a terminal in the parent directory holding trainkit/. A path names a file or directory relative to that location unless it starts from the filesystem root. The output argument names the directory this run’s two current files go into.

The extracted package directory and an output directory. An input CSV file or image folder feeds the trainkit command, which writes two separate files: run.json and model.json.
The runner produces a measurement record and a readable parameter export.

Run the commands below from the directory holding trainkit/. On Windows, write py where they say python3. Check your working directory before pasting a relative input path. The archive also carries the experiment code behind the book’s own runs.

Fit a function and read every update

Start with the twelve-point function fit. Its model predicts ax+b, so the export holds one multiplying weight and one bias. The task is small enough to print every update’s parameters and gradients. It evaluates the same twelve inputs it trains on, and the word “validation” in its output doesn’t make it a held-out test.

Run the fit

python3 -m trainkit fit trainkit/samples/fit.csv --output fit-output

The first printed update is a=0.157576, b=0.200000, grad_a=−1.575758, grad_b=−2.000000. The last exported values are a=1.985458 and b=0.999998. Check the signs: the negative initial gradients produce a positive update when the optimizer subtracts the scaled gradients. The final values approach the sample function’s slope and intercept through the repeated arithmetic of earlier chapters.

Train the sample table

The table command reads four features and a species target from the supplied sample. Read the printed tensor shapes before the losses. Training X is [120,4] with y [120]; validation X is [30,4] with y [30]. The model has 403 parameters and runs four batches per epoch. Those dimensions describe this sample and nothing else. Your own file gives different numbers: the shape’s second number is your feature count after preprocessing, and categorical features expand into one column per training category. The first number counts examples in that split. Classification y has one label per example, and regression y has shape [examples, 1].

Excerpts of the table command’s printed output, labeled: tensor shapes, split, parameter count, final epoch and baseline. Accuracies are printed as decimal fractions, not percentages.
What the sample-table run prints.

Train and compare

python3 -m trainkit table trainkit/samples/iris.csv --target species --output table-output

The last printed epoch is 150, step 600: loss 0.037958, val_loss 0.079900, val_accuracy 0.966667. That is 29 of the 30 validation flowers. Always guessing the most common training species scores 0.333 on the same validation split, so the sample model beat its baseline. These are decimal fractions, not percentages.

Use the other shipped sample formats

The image and sequence commands follow the same pattern: load, describe shapes, train, evaluate, export. Their inputs and objectives differ. Keep the task-specific baseline beside each score and check whether higher or lower is better.

Printed output from the image and sequence commands, each beside its command and split, shown apart from the digit CNN trained earlier in the book.
The command’s default settings differ from the book’s digit CNN, so these are different runs.
python3 -m trainkit images trainkit/samples/digits --output images-output

The image command reports training X [1437,1,8,8], validation X [360,1,8,8], and 666 parameters. Final validation accuracy is 0.953, against a baseline of 0.100. It has the same model family and parameter count as the digit CNN in the convolutional network chapter, but a different split, learning rate, and number of updates, so the two accuracies are not the same measurement.

Train a sequence window

python3 -m trainkit sequence trainkit/samples/sine.csv --target y --window 20 --output sequence-output

This reports training X [780,20,1], validation X [180,20,1], and 273 parameters. Final validation mean squared error is 0.00265, against 0.522 for predicting the mean. Lower is better. The sequence loader uses the target’s history alone and ignores other columns. Its windows respect the chronological split, which is why the input count is below the number of raw rows.

Replace the input path and target

Replace the sample path and name your target before you change anything else. A table needs a header and finite numeric or categorical features, with enough examples per class for the chosen split. Missing, malformed, or nonfinite cells are errors. An identifier that leaks the target is still syntactically valid, so the loader accepting a file replaces none of the split and feature review from earlier chapters.

A measured house-price regression

Use trainkit table house_prices.csv --target price --output house-output. This example uses a synthetic 400-row table with four numeric features and prices in dollars. Its default task detection selects regression. The loader fits the target mean and standard deviation on training rows only, trains on (price − mean) / standard deviation, and converts predictions back to dollars for evaluation. The defaults are 150 epochs, batches of 32, and Adam with learning rate 0.01.

Tensors: train X [320, 4], y [320, 1]; validation X [80, 4], y [80, 1]
Split: 320 training / 80 validation; 10 batches per epoch
0.weight                 [16, 4]                 64 parameters
0.bias                   [16]                    16 parameters
2.weight                 [16, 16]               256 parameters
2.bias                   [16]                    16 parameters
4.weight                 [1, 16]                 16 parameters
4.bias                   [1]                      1 parameters
Total parameters: 369
epoch 150.000 step 1500: loss 113131112.000000, val_loss 122866216.000000
Final evaluation: {'loss': 113131112.0, 'val_loss': 122866216.0, 'rmse': 10636.311014632845, 'mae': 8602.892578125, 'r2': 0.9941526565674457, 'val_rmse': 11084.503416933028, 'val_mae': 8895.0078125, 'val_r2': 0.9933911835403447}
Baseline: {'kind': 'training mean', 'prediction': [395236.625], 'mse': 18592370688.0, 'rmse': 136353.84368619756, 'mae': 117319.1875}
Regression units: loss/val_loss and baseline MSE are target units squared; RMSE/MAE are target units.
Validation R²: 0.993391
beats baseline: yes (by 99.34%)

This excerpt keeps the opening inventory and final evaluation; the CLI also prints each epoch’s loss line. Tensors gives the input X and target y shapes; Split gives the training and validation counts and batches per epoch. Each named weight or bias line gives its shape and scalar parameter count; Total parameters sums them. An epoch counts passes through training examples; a step counts optimizer updates. loss and val_loss are training and validation mean squared error (MSE), in dollars squared here.

beats baseline: yes/no (by x%) compares validation MSE: x = 100 × (1 − model MSE / baseline MSE). A negative percentage means worse; a zero baseline has no defined percentage improvement and cannot be beaten. This run reduced validation MSE by 99.34%. Your result depends on your data. Classification instead prints cross-entropy loss, accuracy/val_accuracy as fractions correct, and the training-majority class prediction and its validation accuracy. The small fit command additionally prints a and b (weight and bias), grad_a and grad_b (their batch gradients before the update), then the full named weights and gradients dictionaries.

Read what the model learned

An export is a saved representation of values and their metadata. Open model.json to read the parameters and the preprocessing that gives them meaning. Open run.json in the dashboard for training curves and recorded states. The two files have different schemas because they answer different questions. The readable parameter export is not a complete optimizer-resume checkpoint.

The four first-row weights under the four standardized input features, followed by the sum, the bias and tanh.
A weight row specifies how one hidden unit combines its preprocessed inputs.

The sample table’s export has format trainkit-model-v1. Its first tensor, 0.weight, has shape [16,4]. The first row is [−0.355,−0.579,0.531,0.197]. In order those multiply standardized sepal_length, sepal_width, petal_length, and petal_width. Then 0.bias[0] is added and tanh applied. Those four numbers are one hidden unit’s input multipliers, not four class probabilities and not four feature-importance scores.

python3 -c 'import json; m=json.load(open("table-output/model.json")); print("format:",m["format"]); print("parameters:",[(p["name"],p["shape"]) for p in m["parameters"]]); k=m["parameters"][0]["name"]; print("first tensor:",k); print("first row:",m["weights"][k][0])'

Where this shows up when you train

For your own run, keep the command, the input and split description, the final metric and baseline, the first weight row, and a sentence saying what each row entry multiplies. If the model misses the intended criterion, run the diagnostic checks from the previous chapter and keep that result in the record.

What you now know

  • The CLI reports data shapes, parameter shapes and evaluation.
  • The sample result and the baseline must use the same split.
  • An exported weight row specifies input multipliers for a unit.

Where we’re headed

The final chapter gathers the terms and notation, so any definition is one page away while you work. Continue the story.