DATA1002 Cheatsheet · Informatics: Data and Computation

0 · Exam map unit outline

Written Examination 50% · Weekly Coding Tasks 10% · Early-feedback MCQ 2% · Practice Coding Test 0% · Project Stage 1 Report 20% · Python Coding Test 10% · Project Stage 2 Poster 8%.

permitted-materials truth · DATA1002 Semester 2 2026 Unit Outline, assessment table“Written exam — Exam CLOSED book exam with MC and short essay questions”

Priority: definitions → relationship → quantities → mechanism → boundary. The final is the 50% row: two-hour MCQ and short essay, with a 40% minimum.

1 · Frame the decision C1

Data science lifecycle = iterative question framing, acquisition, cleaning, analysis, modelling, communication and review. It is not a linear software project: iteration, evidence and decision use come before modelling.

Decision question = bounded question naming actor · outcome · evidence · action. A project failure mode prevents valid, usable or maintainable results.

prediction erroreᵢ = yᵢ − ŷᵢ · observed minus predicted; sign shows over/under-prediction only after the target and scale are named.

1b · Reproducible synthesis C12 · course data

Reproducible analysis means data choices, code, outputs and interpretation can be rerun and inspected. A poster narrative orders question → data → method → result → evaluation → limitation.

2 · Values, types, contracts C2

Python value = a concrete object: number, string, Boolean or collection. Its data type fixes representation, valid operations and interpretation. Trace what reads, changes and depends on it.

function contracty = f(x) · specified typed input x → observable output y under documented type and boundary conditions.

Automated test = reproducible comparison of behaviour/output with an expected result for specified input. State the expected result before execution. Change one condition at a time so a mismatch localises the defect.

2b · Trace, do not guess C2 · C3

Minimal example: expose one rule; explain why it works; name what breaks it; state failure behaviour. One success is not evidence for other types, missing values or boundaries.

Written-answer stem → “For input x of type …, the condition … selects …; the observable result is …; the boundary case changes this because …”.

2c · Contract failure grid C2

3 · Branch, iterate, parse C3

Conditional branch selects statements using a Boolean condition. Iteration repeats over a sequence or while a condition holds. Text parsing extracts structured information from character data using explicit rules.

loop accumulationsₖ = sₖ₋₁ + xₖ · define the initial value and invariant: after k items, what exactly does sₖ mean?

Invariant test: choose an initial value that makes the claim true at k=0; show one update preserves it. A plausible final total cannot repair a wrong starting state.

4 · Choose the structure C4

List = ordered mutable collection accessed by position. Dictionary = mapping from unique keys to values for direct lookup/update. Aggregation = combine observations into grouped counts, totals, means or rates.

grouped meanx̄g = (1/ng) Σi∈g xᵢ · maintain both sum and count for each key; divide inside that group.

5 · Cleaning as an audit C5

Data provenance records where data came from and how they were collected, transformed and used. A missing-value convention distinguishes absent, unknown, inapplicable or withheld values. A data-quality rule tests validity, consistency, completeness, uniqueness or timeliness.

missingness ratermiss = nmissing / nrows · numerator and denominator refer to the same in-scope field and records.

Boundary: the percentage quantifies recorded absence, not its cause. Clean-looking data need not be representative, correctly labelled or fit for the decision.

5b · Quality decisions C5

10 · Annotation before modelling C10

Natural language processing represents, analyses or generates human-language data. Data annotation assigns labels or structured information. Inter-annotator agreement measures consistency among people using the same scheme.

observed agreementPo = nagreements / nitems

6 · Reusable data work C6

Python function = named reusable block with inputs, operations and optional return. Module = reusable file/library namespace. DataFrame = labelled two-dimensional table whose columns may have different types.

row-count invariantnout = nin − nremoved + nadded

Record counts before/after every filter and join. A filter should lose only rows failing its predicate. A join may duplicate keys: unexplained change → inspect key uniqueness and join type before continuing.

Limit: modularity improves testability but does not validate the semantics of a column, transformation or external source.

7 · Visual comparison C7

Visual encoding maps data to position, length, colour, shape or another channel. A comparison baseline is the reference value/group. Chart integrity means scale, selection and decoration do not misrepresent data or uncertainty.

normalised raterg = xg / ng · numerator and denominator must describe the same population and period.

Choose the chart after naming the comparison. Show denominator/rate unit and evidence boundary. Salience can amplify a biased comparison; polish never repairs poor data.

8 · Similarity → recommendation C8

Clustering groups by defined similarity/distance. A distance metric quantifies feature-space dissimilarity. A recommendation ranks suggestions from user/item/similarity information and an objective.

Euclidean distanced(x,z) = √[Σⱼ₌₁ᵖ (xⱼ − zⱼ)²]

Path: define p features → justify/standardise scales → difference, square, sum, square-root → rank neighbours → test usefulness for the decision. A cluster is not a natural label; it is an output of representation, scaling, metric and algorithm choices.

9 · Predict and evaluate C9

Predictive model = fitted rule mapping features to an unknown outcome for new observations. Evaluation metric quantifies one aspect of performance. Generalisation is performance on relevant new data beyond training examples.

classification accuracyAccuracy = (TP + TN) / (TP + TN + FP + FN)

Training accuracy and one aggregate score can hide overfitting, subgroup error and an unusable threshold.

11 · AI reliability and fairness C11

Generative model produces from learned patterns/context. Algorithmic fairness examines group errors, benefits and burdens. Human oversight checks evidence and consequences before action.

selection-rate gapΔr = rA − rB · diagnostic signal, not a complete fairness judgement or remedy.

12 · Worked 1: cleaning bug course worked example · 6 marks

Given: a CSV stores missing ages as −1 and prices as strings such as '$1,240'. Outline a testable pipeline before model fitting.

1. Inspect types, ranges and missing conventions. [1]
2. Replace documented age sentinel −1 with missing. [1]
3. Remove currency symbols/separators; convert price; expose invalid conversion. [1]
4. Record row counts and summaries before/after each transform. [1]
5. Test plausible ages and numeric nonmissing prices. [1]
6. Fit only after documenting how missing ages are excluded, imputed or represented. [1]

answer to the end−1 is no longer evidence of a young age; '$1,240' becomes numeric 1240 only after documented character removal; failures stay visible; model input and exclusions are auditable.

12b · Audit questions C5 · C6

Record the first transition where actual state diverges from the expected state.

12c · Formula sprint C1–C12

13 · Worked 2: accumulator C3

Task: verify a text-processing loop that aggregates accepted values.

relationshipsₖ = sₖ₋₁ + xₖ

Conclusion: correctness follows from a true base case plus an update that preserves the stated meaning of sₖ; a plausible final total alone proves neither.

14 · Worked 3: grouped mean C4

relationshipx̄g = (1/ng) Σi∈g xᵢ

1. Choose a dictionary because the group is a key.
2. For every record in group g, update both sum[g] and count[g].
3. Compute x̄g only from the matching sum and ng.
4. Check by reconstructing sum[g] = x̄g × ng; then interpret “per observation in group g”.

Trap → Ranking raw totals rewards group size; it answers a different question from a group mean or rate.

14b · Row-count drill C6

relationshipnout = nin − nremoved + nadded

15 · Worked 4: distance C8

Task: explain a cluster or recommendation from two represented observations.

relationshipd(x,z) = √[Σⱼ₌₁ᵖ (xⱼ − zⱼ)²]

1. Name every feature j and its unit/meaning.
2. For each coordinate, compute xⱼ−zⱼ; square it so signs do not cancel.
3. Sum across the p features; take the square root.
4. Repeat only after scaling is justified; then rank neighbours or explain assignment.
5. Evaluate usefulness against the decision—not resemblance alone.

answer boundary“These are nearest under this representation, scaling and Euclidean metric; the result is not a discovered natural category.”

16 · Visual judgement cards C7

Trap → A polished chart can make a weak or biased comparison more salient; it cannot make the comparison valid.

16b · Rate self-test C7

relationshiprg = xg / ng

1. Define xg as the counted event/quantity for group g.
2. Define ng as the matching in-scope population or exposure for the same period.
3. Compute within each group before comparison.
4. Show the rate unit or denominator with the visual.

17 · Worked 5: model evaluation C9

Task: evaluate a classifier without mistaking training fit for useful performance.

relationshipAccuracy = (TP + TN) / (TP + TN + FP + FN)

1. Keep training examples for fitting; use held-out evidence for evaluation.
2. Populate TP, TN, FP and FN; denominator = all held-out cases.
3. Compute accuracy; compare with class balance and the decision baseline.
4. Judge unequal consequences of FP versus FN and inspect subgroup error.
5. Conclude only about relevant new data represented by the evaluation set.

answer boundaryAccuracy counts correct predictions but can conceal class imbalance, unequal error costs, subgroup failure and an unusable threshold.

18 · Annotation diagnosis C10

relationshipPo = nagreements / nitems

Trap → Observed agreement is easy to compute but does not adjust for agreement expected from label frequencies.

19 · Method chooser C1–C11

20 · Classic traps all chapters

21 · AI decision audit C11

selection-rate diagnosticrA = selectedA / eligibleA · rB = selectedB / eligibleB · Δr = rA − rB

1. Give each numerator and eligible-population denominator.
2. Subtract rates using an explicit group order so the sign is interpretable.
3. Inspect uncertainty and whether comparison groups are relevant.
4. Examine error, benefit and burden distributions—not only aggregate performance.
5. Assign accountable review before action; document evidence and limitations.

Conclusion: the gap is a diagnostic signal. It cannot alone establish validity, explain mechanism or choose an ethical remedy. Fluent output is not verified evidence.

23 · Exam-morning plan retrieval control

1. Blank page: write the eleven relationships; define denominators.
2. Say aloud: lifecycle, provenance, contract, invariant, baseline, generalisation, oversight.
3. Changed case: run the chooser; show mechanism and boundary.
4. Error log: correct the first unsupported inference.
5. Check: precise definitions, signs/orders, units, denominators and evidence-bounded claim.

answer skeletonDefine → relationship → quantities → transformation → independent check → task interpretation → boundary.