STAT405: pass the exams, not just read the notes
Your complete guide to University of Wisconsin-Madison's data science computing project course. See where the marks are, work real practice questions, and study with an AI tutor that knows STAT405.
Sia generates STAT405 practice questions, walks through linux and the emacs editor step by step, and quizzes you on the material the exam weights most heavily.
Find what is wrong
An HTCondor submit file processes five text files in parallel. Which line makes each job receive its own input file?
executable = count.sh
arguments = $(Process)
transfer_input_files = part$(Process).txt
output = out$(Process).txt
queue 5queue 5 creates five jobs with $(Process) equal to 0, 1, 2, 3 and 4.
The output line likewise gives each job its own output file, but it is the input line that assigns the data.
This is the pattern in the course's word-counting exercise: split one large file into pieces, then queue one job per piece.
The trap: Assuming HTCondor jobs share a filesystem. Each job only sees the files you tell it to transfer. classic slip!
One exam decides 50% of your grade. No make-up except a documented serious problem. This whole page is built around that.
Overview
What STAT405 is, and where it sits
STAT 405 Data Science Computing Project (cross-listed as STAT 605 for graduate students) is UW-Madison Statistics' course on computing at a scale a laptop cannot handle. In Fall 2026 it is taught in person by John Gillett with teaching assistant Ming Pei, from a public course page that lays out every homework and the point budget.
The first half builds tools: the Linux command line and shell scripts, the emacs editor, git and GitHub, then R driven by the Slurm scheduler on the Statistics High Performance Computing cluster. The second half moves to distributed high-throughput computing with HTCondor at UW's Center for High-Throughput Computing, where the signature homework searches 2.4 million spectra for galaxies. Teams then propose, run, report and present a data analysis of their own on a 10 to 100 GB dataset.
Grading is about 200 points: roughly 25 for in-class group exercises and quizzes, 40 for four homeworks, 100 for two in-class exams on 8 October and 19 November 2026, and 35 for the group project. Grades are the higher of a percentage scale and a percentile scale awarding A to the top 40%.
Always treat your own course outline and the exam timetable as authoritative.
Difficulty & time commitment
Is STAT405 hard, and how much time does it take?
STAT405 is manageable if you keep a weekly rhythm and treat the back half as the main event. The pattern is consistent: it starts gently and steepens, and the heaviest assessment is the part that separates grades.
The difficulty curve and the assessment weighting point the same way: the back half is harder and worth more. Front-loading effort there is the highest-return decision in the course.
Is this course for you
Who tends to do well, and who tends to struggle
You will likely do well if
- You want to run computations on a cluster, not just on your laptop.
- You are comfortable typing at a terminal and reading error messages.
- You can commit to a team project with staged deadlines.
- You have basic R and one programming course behind you.
You may struggle if
- You want more statistics; this is a computing course.
- You copy submit scripts without understanding them; the exams test understanding.
- You are late with project components; they earn no credit late.
- You miss class; groupwork points come from being at the table.
- Keep a personal command cheat-sheet for shell, git, Slurm and HTCondor as you go.
- For each homework, write a one-paragraph explanation of what every script line does before the exam.
- Pick a project dataset that is many small files; it maps cleanly onto HTCondor.
- Automate the whole pipeline; the report and the course page both reward reproducibility.
Syllabus
The 8 topics, block by block
The exam-weight marker on each topic shows where the marks concentrate. The amber topics carry the highest exam weight.
T1 · Linux and the shell
Learning outcome 1; schedule blocks 1-2Files, permissions, pipes, wildcards and shell scripts.
T2 · The emacs editor
HW1Editing local and remote files; HW1.
T3 · git and GitHub
Learning outcome 1Commits, branches, collaboration and tracking change.
T4 · R for large computations
HW2Scripts that read many files, loops and functions; HW2 galaxy search.
T5 · Slurm and the Statistics HPC cluster
HW3Submit scripts, job arrays and collecting output; HW3 flights from Madison.
T6 · HTCondor and the CHTC
HW4Submit files, .sh wrappers, thousands of parallel jobs; HW4 2.4 million spectra.
T7 · Designing a parallel data analysis
ProjectChoosing a dataset, splitting the work, automating download to model.
T8 · Reporting and presenting
ProjectA 600-word report with graphics and a peer-graded talk.
How it's assessed
Assessment structure
| Component | Weight | Format & timing |
|---|---|---|
| In-class group exercises and quizzes (drop lowest two) | 12.5% | Random tables of six solve practice questions, then each student completes an individual quiz. Most class meetings. Lowest two dropped. |
| Homework HW1-HW4 (emacs, R, Slurm/HPC, HTCondor/CHTC) | 20% | HW1 emacs (8 pts), HW2 galaxy search in R (8), HW3 Madison flights on the Statistics HPC with Slurm (8), HW4 2.4 million spectra at CHTC with HTCondor (16). Spaced across the semester. 1% per hour late penalty. |
| Two in-class exams (8 October and 19 November 2026) | 50% | Two in-class exams on Linux, R, git, Slurm and HTCondor; the split between the two is not published. Thursday 8 October and Thursday 19 November 2026. No make-up except a documented serious problem. |
| Group data science project: proposal, presentation, peer feedback, 600-word report | 17.5% | Group of 4-5: proposal with data-loading code, staff meeting, in-class presentation graded by peers, peer feedback, 600-word report. Final weeks. Late project work not accepted. |
- The four components sum to 100 and there is no separate hurdle. Your grade is the higher of the percentage scale (A 92+, AB 88, B 82, BC 78, C 70, D 60) and the percentile scale (A top 40%, AB next 20%, B next 20%). Homework late penalty is 1% per hour; project components have no late credit.
- Two in-class exams, Thursday 8 October and Thursday 19 November 2026, together worth 100 of about 200 points; the course page does not publish the split between them. Expect questions on shell commands, git, Slurm and HTCondor scripts and R code from the homeworks.
This is an exam-cram course. With the exams at 50% of the grade and the two in-class exams (8 october and 19 november 2026) alone at 50%, your result is overwhelmingly decided by how well you perform under time pressure. No make-up except a documented serious problem.
Final exam timing: During the examination period. Confirm the exact date and venue on your exam timetable.
How to actually pass it
A weekly rhythm, two checklists, and the traps to avoid
The course rewards consistency over cramming, and practice over re-reading. Here is the loop that works, then what to have nailed before each exam.
The weekly loop
Before the mid-semester checklist
- Write a shell script with wildcards, pipes and a loop.
- Edit and save a remote file in emacs.
- Clone, commit, push and resolve a simple conflict in git.
- Write an R script that processes many input files.
Before the final heaviest topics
- Write a Slurm submit script with a job array and collect output.
- Write an HTCondor submit file and wrapper script for thousands of jobs.
- Explain how to split a large file for parallel processing.
- Describe your project pipeline from download to result.
The mistakes that cost marks
Large files on the login node. The project instructions say to download per job and remove afterwards; HTCondor copies files back otherwise.
Hard-coded paths. Scripts that run on your laptop but not on the cluster.
Unequal project contributions. Presentations are peer-graded; make the division of work visible.
Teaching team
Who teaches STAT405
The bios below are factual. We do not rate lecturers; any star ratings are submitted by students who have taken STAT405.
Teaching team as listed in the course materials reviewed. AskSia does not rate lecturers; star ratings are submitted by students who have taken STAT405.
Formula & concept sheet
The vocabulary and formulas you must own
- Shell script
- A file of commands run by bash.
- Pipe
- Sending one command's output into another's input.
- Commit
- A recorded snapshot in git.
- Slurm
- The scheduler that queues jobs on the Statistics HPC cluster.
- Job array
- One Slurm submission that launches many indexed jobs.
- HTCondor
- The high-throughput scheduler used at CHTC.
- Submit file
- The HTCondor description of a job's executable, inputs and outputs.
- High-throughput computing
- Many independent jobs run in parallel across many machines.
- Reproducible pipeline
- Scripts that recreate an analysis from raw data without manual steps.
Set texts
The prescribed reading
The syllabus references map straight onto these.
The Linux Command Line
.
Where it fits
Prerequisites, related courses & why it matters
Prerequisite: STAT 303 or STAT 240, and CS 200, 220 or 300. 3 credits; two 75-minute lectures per week. Cross-listed as STAT 605.
Your STAT405 study toolkit
Study the course with Sia, not just read about it
Each tool already knows STAT405: your syllabus, your texts, and where the marks are. Grouped by how you study, from first contact to exam week.
FAQ
Frequently asked questions
Is STAT 405 hard?
Moderate on the six-factor rubric, driven by workload: real cluster computing, four substantial scripting homeworks and a team project, with a generous curve.
What is the assessment breakdown?
About 200 points: groupwork and quizzes 25, homework 40, two exams 100, group project 35, so roughly 12.5% / 20% / 50% / 17.5%, per the instructor's Fall 2026 course page.
Who teaches it?
John Gillett (Teaching Faculty) with teaching assistant Ming Pei in Fall 2026.
What are the prerequisites?
One of STAT 303 or STAT 240, and one of CS 200, CS 220 or CS 300.
What is the difference between 405 and 605?
Same class and page; 405 is the undergraduate number and 605 the graduate number.
Do I need my own laptop?
Yes, one that can ssh to the course's Linux environments; DoIT lends laptops and the campus labs have ssh clients.
Study STAT405 with Sia
Work through linux, the emacs editor, git and the rest of the course with a tutor that knows it and quizzes you on the topics the assessments weight most heavily.
Start studying with Sia
