Training, weights and checkpoints AI

Separating model updates from responses to one fixed input.

Weights and biases are model parameters. Activations are values produced when those parameters process a particular input. Training changes parameters; evaluating a fixed input at a saved checkpoint reveals the resulting activations and prediction.

Both pilots minimize cross-entropy. For a single labeled example without class weighting, this is $-\log p_y$, where $p_y$ is the probability assigned to the recorded class. PyTorch computes it directly from logits; an extra softmax should not be applied before its CrossEntropyLoss.

PyTorch computes gradients through the model; AdamW uses them to update parameters. A batch is one group of examples used for an update. An epoch is a pass through the training set. A checkpoint saves a model state at a chosen point; the curves join discrete observations, not continuously measured recovery.

The GRU saves epochs 0, 1, 2, 3, 5, 10, 15 and 20. Epoch 0 is untrained; epoch 15 is selected by validation macro F1; epoch 20 is the last training snapshot. “Selected,” “last,” and “best on a new population” are different claims.

Explore the neural-network demo.