INFS3822 Chap.5 Data Preparation and Feature Engineering
Data Preparation and Feature Engineering
Data preparation controls what a model is able to learn. This chapter distinguishes cleaning from deleting difficult cases, explains one-hot and ordinal encoding, scaling and transformation, and separates feature creation, selection and dimensionality reduction. It applies the central anti-leakage rule: split first, learn transformations on training evidence, and apply them unchanged to testing and future observations.
What this chapter covers
- 01
Missing values and valid outliers
- 02
One-hot and ordinal encoding
- 03
Min-max and standard scaling
- 04
Log transformation
- 05
Feature creation and selection
- 06
Principal component analysis
- 07
Training-only transformations
Worked example · free
Prepare a mixed customer table
- +1One-hot encode nominal channel and justify ordinal encoding or indicators for tier.
- +1Create recency from dates using only information available at the decision time.
- +1Inspect spend and consider a log transform or robust scaling for the skewed distribution.
- +1Split before fitting imputers, scalers, feature selectors or dimensionality reduction.
Key terms
- One-hot encoding
- Representing nominal categories with separate binary indicator columns.
- Ordinal encoding
- Mapping categories to ordered numeric levels when a genuine order exists.
- Feature selection
- Choosing a subset of relevant inputs through filter, wrapper or embedded methods.
- Dimensionality reduction
- Creating a lower-dimensional representation that retains useful variation or structure.
Data Preparation and Feature Engineering FAQ
Why must preparation be fitted on training data?
Imputation values, scaling parameters, selected features and components are learned from data. Letting the test set influence them leaks information from supposedly unseen evidence and produces an optimistic evaluation. Fit on training rows, freeze the transformation and apply it unchanged to testing rows.
When should I use one-hot encoding?
Use it for nominal categories that have no meaningful order, such as channel or colour. Assigning integers would imply greater-than relationships that do not exist. For truly ordered categories, ordinal encoding may be justified, although indicators remain useful when spacing between levels is not meaningful.
Are all outliers errors?
No. Some are recording mistakes; others are rare but valid observations that may matter most to the decision. Investigate origin and context before removing them. Report the rule, compare sensitivity and avoid improving model scores by deleting legitimate difficult cases.
What belongs in a feature contract?
Record source fields, types, units, allowed categories, missing-value meaning, time cut-off, transformation formula and output order. Save fitted encoders, imputers and scalers with the estimator. Test ordinary, missing, boundary, rare and unseen-category rows through training and deployment implementations so a schema or ordering difference is found before it corrupts live predictions.
Assessment move
Create a preparation ledger with variable type, missingness, decision-time availability, transformation and reason. Draw the train/test boundary before any learned step. For every engineered feature, explain what source fields it uses and whether those fields existed at prediction time. Rehearse one leakage diagnosis in each practice session.
Work through a mixed table by classifying each operation as a fixed rule or a learned transformation. Parsing a documented date format can be fixed; estimating an imputation value, scaling range, selected feature set or principal component learns from data and must be fitted inside the training boundary. For nominal categories, explain why indicator columns avoid invented order.
For ordinal categories, state the order and question whether numeric spacing is meaningful. For skewed numeric data, inspect the distribution before proposing a log or robust transformation and keep the target units visible when interpreting results. Build a timestamp audit beside the variable ledger. Record when each field is created, when the prediction is required and whether the field would be available for a new case.
Derive one feature manually from source fields, then reproduce it through the pipeline. Include missing values, a rare valid category, an unseen category and a boundary value in a small reference set. This catches differences in mapping, feature order and time handling that aggregate accuracy may conceal.
Practise split order with a written sequence: reserve final evidence, fit preparation on training rows, apply the fitted objects unchanged, compare candidates through validation, lock the completed procedure, then open the final set once. When using cross-validation, remember that every learned preparation step belongs inside each training fold.
Challenge common instructions such as remove outliers, fill missing data and reduce dimensions by asking what evidence justifies them and what information may be lost. A rare case may be precisely the event the model must handle. End each revision session by diagnosing one suspicious result: implausibly high performance, a category failure after release, or a test distribution that looks identical to training.
Trace it through target leakage, split contamination, schema mismatch and transformation drift before changing the estimator.
Working through Data Preparation and Feature Engineering in INFS3822? Sia is AskSia’s AI Computer Science tutor — ask any INFS3822 Data Preparation and Feature Engineering question and get a clear, step-by-step explanation grounded in how INFS3822 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
