ELEC5622 Chap.10 Feature Extraction with PCA, Shape Models and FLDA
Feature Extraction with PCA, Shape Models and FLDA
This chapter addresses the small sample-size problem common in medical imaging, where features outnumber patients. PCA finds orthogonal directions of maximal variance through the eigenvectors of the covariance matrix, and a variance threshold chooses how many to keep. Active shape models apply PCA to landmarks, and Fisher's linear discriminant chooses a projection that separates labelled classes.
A five-point example is worked through covariance, eigenvalues and projection by hand. Read the chapter with Project 1 in mind: forty subjects described by ninety regional volumes is exactly the shape of data these methods were designed for. The lecture's knee vibration example shows FLDA raising accuracy to 72% with six combined features.
What this chapter covers
- 01
The small sample-size problem
- 02
Feature extraction versus feature selection
- 03
PCA as variance maximisation and eigendecomposition
- 04
Choosing the number of components
- 05
Reconstruction from principal components
- 06
Active shape models and point distribution models
- 07
Fisher linear discriminant analysis
Worked example · free
Components needed for a variance target
- 1Total variance = 4.5 + 1.2 + 0.3 = 6.0.
- 1PC1 keeps 4.5 / 6.0 = 75%; PC1 and PC2 keep 5.7 / 6.0 = 95%.
- 1One component falls short of 90% and two exceed it, so k = 2.
Key terms
- Principal component
- A principal component is an eigenvector of the data covariance matrix, ordered by the variance it carries.
- Active shape model
- An active shape model is a statistical model of landmark positions built by PCA and fitted to new images.
- Fisher criterion
- The Fisher criterion divides the spread between projected class means by the spread within the projected classes.
- Point distribution model
- A point distribution model represents a shape by corresponding landmark points and their statistical variation.
- Dimensionality reduction
- Dimensionality reduction maps data to fewer features while keeping the information a task needs.
Feature Extraction with PCA, Shape Models and FLDA FAQ
What is the small sample-size problem?
It arises when a study has far fewer training cases than feature dimensions. Model parameters cannot then be estimated reliably, which is why dimensionality reduction by extraction or selection comes before classification.
How is PCA different from FLDA?
PCA ignores labels and keeps the directions of greatest variance. FLDA uses class labels and picks the projection that maximises the gap between class means relative to the spread within classes, which suits classification better.
How many principal components should be kept?
The usual rule keeps the smallest number whose eigenvalues account for a chosen fraction of total variance. The active shape model slides keep most of the variance, quoting 98%, and drop the small remaining components.
Why must data be centred before PCA?
PCA describes spread about the mean, and the covariance matrix is built from deviations from it. Without centring, the first direction tends to point at the mean itself rather than along the main axis of variation.
What are the weaknesses of active shape models?
The lecture lists two: the fitting optimisation can be trapped in local minima, and reinitialisation is problematic. The strengths are a shape prior that resists segmentation errors and fast optimisation.
What is the difference between feature extraction and feature selection?
Feature extraction builds new features by combining the originals, as PCA does with its linear combinations. Feature selection keeps a subset of the original features unchanged. Both reduce dimension to ease the small sample-size problem.
Exam move
Compute a 2 × 2 covariance matrix, its eigenvalues and its first eigenvector by hand at least twice, checking that eigenvalues sum to the trace. Practise choosing k from an eigenvalue list for different targets. Then describe the active shape model steps in order and contrast PCA with FLDA in two sentences.
Practise one FLDA comparison of two projections using the Fisher ratio, and explain why the direction with the largest mean gap is not always the winner. Before the exam, be able to state the PCA objective, maximise projected variance with a unit-length direction, and the Fisher objective, maximise the between-to-within ratio, in one line each.
Working through Feature Extraction with PCA, Shape Models and FLDA in ELEC5622? Sia is AskSia’s AI Electrical Engineering tutor — ask any ELEC5622 Feature Extraction with PCA, Shape Models and FLDA question and get a clear, step-by-step explanation grounded in how ELEC5622 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
