IS6335 Chap.10 Computer Vision and Deep Learning for Images
Computer Vision and Deep Learning for Images
Why an advanced module sits inside a visualization course
Module three covers computer vision applications in week ten, the motivation for taking up deep learning, and augmented and virtual reality, then image classification, neural network architecture, convolutional networks and image captioning with recurrent networks in week eleven.
The tutorials move from pandas, numpy and scikit-learn to model training and captioning in Python, and the named reference for the module is a standard text on deep learning with Python.
The syllabus makes the connection explicit in its third intended learning outcome, which asks you to use visualization to augment machine learning and statistical analysis in applications. That relationship runs both ways.
An image is a continuous field in the framework sense, a regular grid of positions each carrying attributes, so computer vision is what happens when the week six dataset type becomes the input to a model rather than to a chart.
And a model's own behaviour, its errors, its confidences and the features it has learned, is itself data that has to be visualized before anyone can trust it.
Keeping the vocabulary straight
A convolutional network detects local patterns and builds larger ones by repeating that detection over successive stages, which is why it suits images: the structure of the method matches the structure of the data.
A recurrent network produces a sequence, which is why captioning pairs the two; the convolutional part sees and the recurrent part writes. Augmented and virtual reality appear in the same module because they are delivery surfaces where these outputs are consumed, and the syllabus abstract mentions students experiencing virtual reality applications directly.
Two consequences follow from an image being a field.
Position is not available as an encoding channel, because it is already carrying the image, so any derived quantity has to be drawn as an overlay or beside the picture.
And resolution is inherited rather than chosen, so a pattern smaller than a cell is not in the data at all, however confident a model sounds about it.
What a trained model licenses you to say
For this course the interesting question is not how to train a classifier but what its output supports.
A single accuracy number is the least informative summary available, for the same reason a correlation coefficient was in week one: it is a summary whose shape has been discarded.
Overall accuracy does not tell you which classes fail, or whether a rare class fails completely; the view that recovers that is a confusion matrix.
A confident prediction does not tell you whether the confidence is calibrated; the view that recovers that is accuracy plotted against predicted confidence. A training curve alone does not tell you whether a gain generalises; drawing training and validation on the same axes does.
And a class label per image says nothing about what inside the image the model responded to, which a saliency overlay on individual examples begins to answer.
The confusion matrix is the week nine idiom again
A confusion matrix has true classes on one axis, predicted classes on the other and counts in the cells, so it is structurally the adjacency matrix from the previous chapter in a different costume.
It rewards the same move. Reorder the classes so that confusable ones sit together and the block structure of the model's errors becomes visible. A model that confuses two classes symmetrically has a different fault from one collapsing everything onto a dominant class, and both look identical in an accuracy figure.
Class balance is the first thing that matrix settles.
If the positive class is six per cent of the sample, a model that predicts the negative class every time scores ninety-four per cent, so the headline number cannot distinguish a working model from a constant one. The second thing it settles is which error is expensive, because a missed case and a false alarm are rarely symmetric and the relevant figures are the two off-diagonal cells separately rather than their total.
What this chapter covers
- 01
Where module three connects back to the framework
- 02
An image as a continuous field, and what that fixes
- 03
Convolutional stages and the recurrent tail
- 04
Augmented and virtual reality as delivery surfaces
- 05
Why one accuracy number is the weakest summary
- 06
Confusion matrix, calibration plot, training curve, saliency
- 07
Class balance and asymmetric error cost
Answer a classification claim with a view
- 2Ask about class balance and name the view.
- 2Ask which error is expensive and name the figures.
- 2Ask about calibration and name the view.
Key terms
- Confusion matrix
- A grid of true classes against predicted classes with counts in the cells. It is the adjacency matrix idiom applied to a model, and it rewards the same reordering.
- Convolutional network
- A model that detects local patterns and builds larger ones by repeating the detection over successive stages. Its structure matches the grid structure of an image.
- Calibration
- Whether a model's stated confidence matches how often it is right at that confidence. It is recovered by plotting accuracy against predicted confidence, never from an accuracy figure.
- Class imbalance
- A large difference in how often the classes occur. It makes overall accuracy uninformative, because predicting the common class always can score close to the headline number.
Computer Vision and Deep Learning for Images FAQ
We are not machine learning students. How much of this do we need?
Enough to read a model's output as data rather than as a verdict, which is what the third intended learning outcome asks for when it names using visualization to augment machine learning and statistical analysis.
The course introduces the architectures so that the vocabulary is available, and the transferable skill is the auditing one: knowing that an accuracy figure hides class balance, that a confusion matrix recovers it, and that the same reordering move from the network week applies. Training a model well is not what a visualization report is marked on; showing what it does and does not support is.
What is the smallest honest visual deliverable if we include a model in the project?
Three views. A reordered confusion matrix, which says what the model actually does across classes. A training and validation curve drawn on shared axes, which says whether the gain generalised. And a small panel of individual cases the model got wrong, which says what it responds to and is usually the most informative of the three for a reader.
A single accuracy figure answers none of those questions, and a screenshot of a training log answers a question nobody asked.
Assessment move
Work backwards from the audit rather than forwards from the model. Before training anything, write down the confusion matrix you expect, including which two classes you think will be confused and in which direction. Then compare it with the one you get. The gap is the finding, and it is far more interesting to a reader than the accuracy figure, which is the number every other group will also report.
Working through Computer Vision and Deep Learning for Images in IS6335? Sia is AskSia’s AI Information Technology tutor — ask any IS6335 Computer Vision and Deep Learning for Images question and get a clear, step-by-step explanation grounded in how IS6335 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
