MAIE5421 Chap.10 Building a PyTorch Training Pipeline
Building a PyTorch Training Pipeline
Six components of a PyTorch pipeline
The lecture's PyTorch walkthrough uses MNIST digit recognition to build a complete pipeline in six parts: load the data, define the model, train it, evaluate it, tie the parts together and make predictions.
At each stage a design question has to be answered, from what the dataset and learning target are, through the model structure, to which loss function, optimiser and training loop to use.
PyTorch is installed in Colab with a single pip command, and a notebook bundle accompanies the lecture.
Loading data one sample at a time
Because real datasets can be far larger than memory, the lecture defines a custom Dataset class rather than loading everything at once.
Its init method records the file path, the number of samples and a list of indices; its len method reports the dataset size; and its getitem method reads one line of a CSV file, scales the pixel values into the range 0 to 1 and returns them with the label.
A DataLoader then wraps the dataset and delivers batches, with the batch size and whether to shuffle as its two key settings, and random_split divides the data into training and validation sets.
Defining the model
Two styles are taught. The Sequential API lists layers in order and suits a straightforward chain of linear, activation and dropout layers.
The Functional API defines a class with an init method that creates the building blocks and a forward method that wires them together, which allows branches and concatenation.
The lecture also introduces the shapes of linear, LSTM, two-dimensional convolution and max-pooling layers, and the reshape operation that flattens feature maps.
Loss, optimiser, loop and prediction
Mean squared error, cross-entropy and binary cross-entropy are available as ready-made losses, and a custom loss can be written as a class with a forward method.
Optimisers such as SGD, RMSprop, Adam, Adadelta and Adagrad update the model's parameters. The training loop runs the forward pass, computes the loss, zeroes the gradients, backpropagates and steps the optimiser for every batch, evaluates on validation data without tracking gradients, and saves the best model. Prediction reloads the saved weights, switches to evaluation mode and runs a forward pass without gradients.
What this chapter covers
- 01
The six components of a PyTorch pipeline
- 02
Custom Dataset classes with init, len and getitem
- 03
DataLoader batching, shuffling and random_split
- 04
Sequential and Functional model definitions
- 05
Input and output shapes of common layers
- 06
Ready-made and custom loss functions
- 07
Optimisers and the training loop
- 08
Validation, saving the best model and prediction
Worked example · free
Put the PyTorch training loop for one batch in the right order
- 1Forward pass first: pred = model(x).
- 1Then the loss: loss = loss_fn(pred, y).
- 2Clear old gradients with optimizer.zero_grad(), then compute new ones with loss.backward().
- 2Finally update the weights with optimizer.step(); without zero_grad, gradients from earlier batches accumulate into every update.
Key terms
- PyTorch
- A free, open-source Python library for defining, training and evaluating deep learning models.
- Dataset Class
- A class with init, len and getitem methods that tells PyTorch how many samples exist and how to produce each one.
- DataLoader
- The PyTorch object that wraps a dataset and delivers batches of samples, optionally in shuffled order.
- Batch Size
- The number of samples processed together in one iteration of training.
- Sequential API
- A way of defining a network as an ordered list of layers applied one after another.
- Functional API
- A way of defining a network as a class whose forward method wires its building blocks together, allowing branches.
- State Dict
- The dictionary of a model's learned parameters, which is saved during training and loaded for prediction.
Building a PyTorch Training Pipeline FAQ
What do the three methods of a PyTorch Dataset do?
The init method sets up whatever the other methods need, such as the file path, the number of samples and a list of indices. The len method returns the number of samples, which tells the loader when an epoch ends. The getitem method takes an index and returns one sample, typically the input features and the label, and it is where any per-sample conversion or normalisation happens.
Why does my training batch change every time I run the code?
Because the training loader has shuffle switched on, so it draws the samples in a new random order each epoch, and random_split also chooses the training and validation samples at random. That behaviour is intended, since shuffling helps training. If you need identical runs, for example to compare two models fairly in a report, fix the random seed before creating the split and the loaders.
Should my model end with a softmax layer if I use CrossEntropyLoss?
No. The lecture notes that CrossEntropyLoss passes its input through a softmax internally and expects the indices of the correct classes as targets. If the model already applies softmax, the scores are squashed twice and training slows. Return the raw outputs of the final linear layer for training, and apply softmax separately only when you need probabilities for display or thresholding.
Assessment move
Type the lecture's pipeline from memory in the order dataset, loader, model, loss, optimiser, loop, prediction, and run it on a small subset of the digit data. Then break it on purpose: remove zero_grad, add a softmax before CrossEntropyLoss, swap the shuffle settings. Watching what each mistake does to the loss is the fastest way to recognise it in your own project code.
Working through Building a PyTorch Training Pipeline in MAIE5421? Sia is AskSia’s AI Computer Science tutor — ask any MAIE5421 Building a PyTorch Training Pipeline question and get a clear, step-by-step explanation grounded in how MAIE5421 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.