COMP20008: pass the exams, not just read the notes
Your complete guide to University of Melbourne's elements of data processing unit. See where the marks are, work real practice questions, and study with an AI tutor that knows COMP20008.
Sia generates COMP20008 practice questions, works through them step by step, and quizzes you on the material the exam weights most heavily.
Find what is wrong
You join a transactions DataFrame to a small customers lookup on cust_id. Some transactions have a cust_id that is not present in customers, and after the merge those transactions have silently disappeared from the result. Which single change keeps every transaction?
result = pd.merge(transactions, customers, on='cust_id')
# rows with an unmatched cust_id vanishpd.merge defaults to how='inner', which keeps only rows whose key appears in both frames, so unmatched transactions are dropped.
Unmatched customer columns are filled with NaN, which you can then handle explicitly rather than losing the rows.
how='inner' is the default that caused the bug; sorting does not change which rows a join keeps.
The trap: Assuming a merge keeps all rows by default. pandas uses an inner join unless you set how=, so a naive merge silently drops rows whose key is missing on the other side, the classic data-processing bug behind a shrinking row count. classic slip!
One component decides 50% of your grade. YES. This whole page is built around that.
Overview
What COMP20008 is, and where it sits
COMP20008 Elements of Data Processing is the University of Melbourne's second-year introduction to the practical data-science pipeline, taught in the School of Computing and Information Systems and delivered in Python with pandas. It runs the full workflow: acquiring and wrangling messy data, cleaning and transforming it, record linkage and integration across sources, data privacy and anonymisation, exploratory analysis and visualisation, and an introduction to basic machine-learning ideas.
The subject is hands-on: half the grade is continuous project and data-processing work where you actually manipulate data in code, and half is an end-of-semester exam on the concepts behind it. The recurring skill is reasoning about how a data operation behaves (what a join keeps or drops, how a transformation changes a distribution, why an anonymisation choice does or does not protect privacy) rather than memorising syntax.
Official outline: handbook.unimelb.edu.au · COMP20008 outline. Always treat the official outline and the exam timetable as authoritative.
Difficulty & time commitment
Is COMP20008 hard, and how much time does it take?
COMP20008 is manageable if you keep a weekly rhythm and treat the back half as the main event. The pattern is consistent: it starts gently and steepens, and the heaviest assessment is the part that separates grades.
The difficulty curve and the assessment weighting point the same way: the back half is harder and worth more. Front-loading effort there is the highest-return decision in the unit.
Is this unit for you
Who tends to do well, and who tends to struggle
You will likely do well if
- You are comfortable coding in Python and willing to reason about what each pandas operation does to the data.
- You keep the continuous project work moving weekly rather than batching it before deadlines.
- You can connect a data operation to its consequence (what a join drops, what an anonymisation hides, what a plot reveals).
You may struggle if
- You memorise syntax without understanding join semantics, missing-data handling or transformations.
- You leave the project work late; the applied half rewards steady iteration.
- You skip the concepts (privacy, linkage, evaluation) that the exam tests behind the code.
- For every pandas operation, be able to state what it keeps, drops or changes, not just how to type it.
- Practise the privacy and record-linkage concepts, since they are exam-heavy and easy to under-prepare.
- Treat the project as a reproducible pipeline: clean, transform, analyse, visualise, and justify each step.
Syllabus
The 12 topics, topic by topic
The exam-weight marker on each topic shows where the marks concentrate. The amber topics carry the highest exam weight.
T1 · The Data Pipeline & Data Types
Weeks 1–2. The five-stage data pipeline; numerical vs non-numerical vs could-be-either; continuous/discrete; structured/semi-structured/unstructured; levels of measurement; JSON, XML and HTML.
T2 · Data Cleaning, Imputation & Scaling
Week 2. The six data-quality dimensions; missing/inconsistent/duplicate/outlier problems; imputation (drop, mean/median/mode, model-based); min-max & z-score scaling; sampling.
T3 · Web Crawling & Scraping
Week 3. Crawling (link-following frontier, coverage, robots.txt) vs scraping (targeted field extraction); the seed-URL → fetch → parse → BeautifulSoup → clean-matrix pipeline; the visited-set flaw.
T4 · Text Pre-processing & Lemmatisation
Week 3. Sentence splitting with exceptions; tokenisation & normalisation; lemmatisation vs stemming; stop-word removal; n-grams; why text has hidden structure.
T5 · Bag-of-Words, TF-IDF & Visualisation
Week 4. BoW document/word vectors (order discarded); windowed co-occurrence vectors; Euclidean distance; TF vs TF-IDF for distinguishing topics; chart-type choice and the binning effect.
T6 · Clustering: k-means, VAT & the Elbow
Week 5. k-means (assign/recompute, SSE/inertia); the elbow method (advantage/limitation); outlier sensitivity and mitigation (k-medoids, robust/density methods); the VAT reordered distance matrix.
T7 · Hierarchical Clustering & Dendrograms
Weeks 5–6. Agglomerative clustering; single vs complete vs average linkage; building dendrograms (y-axis = merge distance); cutting the tree for k clusters; NMI for comparing clusterings.
T8 · Correlation, Entropy & Mutual Information
Weeks 5–6. Pearson r (linear only); entropy & conditional entropy (log₂); mutual information & NMI; PCA (principal axes, variance explained, the 95%-on-PC1 caution); high-NMI-but-zero-Pearson (y = x²).
T9 · Supervised Learning: k-NN & Decision Trees
Week 7. Classification vs regression; train/test; k-NN (distance, k, scaling, cold-start/Jaccard pitfalls); decision trees (information gain/Gini, depth); high-cardinality split overfitting.
T10 · Regression, Evaluation & Feature Selection
Week 8. Linear regression (least squares, slope, R²); experimental design; k-fold cross-validation; confusion matrix to accuracy/precision/recall/F1; the precision-recall trade-off; feature selection; data leakage.
T11 · Ethics, IP & Privacy
Week 9. Intellectual property (patent/copyright/trademark, licensing, GDPR); privacy levels; k-anonymity & quasi-identifiers; l-diversity vs the homogeneity attack; global differential privacy; bias & fairness.
T12 · LLMs, Prompting, RAG & Verifiable Data
Weeks 10–12. What makes an LLM generative (embeddings, hallucination); prompts vs fine-tuning; the AUTOMAT prompt schema; the RAG store and content-partitioning; blockchain tamper-evidence and digital-signature verification.
How it's assessed
Assessment structure
| Component | Weight | Format & timing |
|---|---|---|
| Continuous / project assessment (data-processing & analysis projects across the semester, ~60 hrs; submissions ~week 5 and ~week 11) | 50% | Project work applied to real datasets; in the current offering delivered as Assignment 1 (individual: coding & analysis + interactive oral via Zoom + critical-thinking submission) and Assignment 2 (group: report + slides + in-person group oral), plus weekly ungraded formative quizzes. A1/A2 sub-weights subject to confirmation. YES. |
| End-of-semester examination | 50% | Closed-book written exam, 2 hours, during the exam period; a formula sheet is printed on page 1; ~11–12 questions of 2–4 marks each — short-answer + small by-hand calculation + critical-evaluation across the whole syllabus. YES. |
- Pass on a weighted average of at least 50%. No single-component hurdle unless noted; confirm against the official subject page.
This is an exam-cram unit. With the exams at 50% of the grade and the continuous / project assessment (data-processing & analysis projects across the semester, ~60 hrs; submissions ~week 5 and ~week 11) alone at 50%, your result is overwhelmingly decided by how well you perform under time pressure. YES.
How to actually pass it
A weekly rhythm, two checklists, and the traps to avoid
The unit rewards consistency over cramming, and practice over re-reading. Here is the loop that works, then what to have nailed before each exam.
The weekly loop
Before the mid-semester checklist
Before the final heaviest topics
- Revise join semantics and missing-data handling until the row-count behaviour is automatic.
- Drill the privacy and record-linkage concepts, a common exam focus.
- Review visualisation and basic machine-learning ideas at the concept level.
- Practise explaining what a given pandas operation does to a dataset in words.
The mistakes that cost marks
Trusting default joins. pandas merges are inner by default and silently drop unmatched rows. Not checking the join type is the most common data-processing error and quietly corrupts results.
Coding without concepts. The exam tests the reasoning behind the code (privacy, linkage, evaluation). Syntax fluency without the concepts leaves half the marks exposed.
Backloading the project. The 50% continuous work rewards weekly iteration; compressing it into the final week produces a fragile, undocumented pipeline.
Teaching team
Who teaches COMP20008
The bios below are factual. We do not rate lecturers; any star ratings are submitted by students who have taken COMP20008.
Dr Caren Han
Subject coordinator and lecturer for COMP20008 in the School of Computing and Information Systems, University of Melbourne.
Professor Eduard Hovy
Professor in the School of Computing and Information Systems, University of Melbourne; lectures COMP20008.
Teaching team as listed in the unit materials reviewed. AskSia does not rate lecturers; star ratings are submitted by students who have taken COMP20008.
Formula & concept sheet
The vocabulary and formulas you must own
- Inner vs left join
- An inner join keeps only keys present in both tables; a left join keeps all rows of the left table and fills unmatched right columns with NaN. Choosing the wrong one silently changes the row count.
- Min-max normalisation
- x' = (x − min) / (max − min): rescales a feature to the range 0 to 1 so features on different scales are comparable.
- Missing data (NaN)
- Absent values represented as NaN; handled by dropping, imputing (mean/median/mode) or flagging, each with different consequences for later analysis.
- Record linkage
- Matching records that refer to the same real-world entity across datasets using similarity on shared attributes; the basis of data integration.
- k-anonymity
- An anonymisation property where each record is indistinguishable from at least k-1 others on the quasi-identifiers, reducing re-identification risk.
Common acronyms: NaN · CSV · ML · EDA · k-anon.
Set texts
The prescribed reading
The syllabus references map straight onto these.
Python for Data Analysis
Wes McKinney.
Where it fits
Prerequisites, related units & why it matters
Second-year subject; assumes basic programming (introductory Python or equivalent). Check the UniMelb Handbook for the exact prerequisite.
Your COMP20008 study toolkit
Study the unit with Sia, not just read about it
Each tool already knows COMP20008: your syllabus, your texts, and where the marks are. Grouped by how you study, from first contact to exam week.
FAQ
Frequently asked questions
Is COMP20008 hard?
It is a moderate second-year subject. The programming is approachable and there is no single make-or-break component, but you need to understand what each data operation does, not just its syntax, across both the project half and the exam half.
How is COMP20008 assessed?
Roughly half the grade is continuous project and data-processing work and half is an end-of-semester examination. The two balanced halves mean steady project work matters as much as exam preparation.
What programming does it use?
Python with the pandas library for data wrangling, plus visualisation and basic machine-learning tools. The de-facto reference is Wes McKinney's Python for Data Analysis.
What topics does it cover?
Data acquisition and wrangling, cleaning and transformation, record linkage and integration, data privacy and anonymisation, exploratory analysis and visualisation, and an introduction to machine learning.
Do I need prior data-science experience?
No, but comfort with basic Python helps. The subject builds the practical data-processing pipeline from the ground up as a second-year introduction.
Study COMP20008 with Sia
Work through the core topics and the rest of the unit with a tutor that knows it and quizzes you on the topics the assessments weight most heavily.
Start studying with Sia