MAT9004 · Mathematical Foundations For Data Science And Ai
Probability and Bayes
Probability is the language of uncertainty — and of machine learning — and MAT9004 emphasises it. The chapter builds from probability spaces, events and the axioms, through equally-likely outcomes (where counting becomes probability, tying straight back to combinatorics), to the set operations: complement, union, intersection and mutually exclusive events. The engine is conditional probability and independence, which lead to the law of total probability and Bayes' theorem — the signature long-answer of the unit, usually a two-stage problem where you reverse a conditional and watch out for the base-rate trap. The second half introduces random variables, expectation and variance, then the named discrete and continuous distributions with their means and variances (which the formula sheet supplies). Everything is procedural: set up the events, apply the right rule, compute the exact probability by hand.
What this chapter covers
- 015.1 Probability spaces, events and the axioms
- 025.2 Equally-likely outcomes: counting becomes probability
- 035.3 Complement, union, intersection and mutually exclusive
- 045.4 Conditional probability and independence
- 055.5 Law of total probability and Bayes’ theorem
- 065.6–5.7 The signature two-stage Bayes example and the base-rate trap
- 075.8–5.10 Random variables, expectation and variance
- 085.11–5.13 The named discrete and continuous distributions
Worked example: Bayes' theorem (a two-stage problem)
- +1Name the events: D = has the disease, + = tests positive. Given: P(D) = 0.01, P(+|D) = 0.95, and a 90%-accurate negative test means P(+|not D) = 0.10.
- +1Total probability of a positive test: P(+) = P(+|D)P(D) + P(+|not D)P(not D) = 0.95·0.01 + 0.10·0.99.
- +1Compute the denominator: P(+) = 0.0095 + 0.099 = 0.1085.
- +1Apply Bayes' theorem: P(D|+) = P(+|D)P(D) / P(+) = 0.0095 / 0.1085.
- +1Evaluate: P(D|+) ≈ 0.0876, i.e. about 8.8%.
Key terms
- Conditional probability
- The probability of A given that B has occurred, P(A|B) = P(A ∩ B)/P(B). It is how you update a probability once you learn that another event has happened.
- Independence
- Two events are independent when one occurring does not change the other's probability: P(A ∩ B) = P(A)P(B), equivalently P(A|B) = P(A). Do not confuse it with mutually exclusive, which means they cannot both happen.
- Law of total probability
- Breaks a probability across a set of exhaustive, mutually exclusive cases: P(B) = Σ P(B|Ai)P(Ai). It is the standard way to build the denominator of Bayes' theorem.
- Bayes' theorem
- Reverses a conditional probability: P(A|B) = P(B|A)P(A)/P(B). The signature long-answer of the unit, where the denominator P(B) usually comes from the law of total probability.
- Expectation
- The long-run average value of a random variable, E[X] = Σ x·P(X = x) for a discrete variable. The 'centre of mass' of the distribution, used to value games and summarise outcomes.
Probability and Bayes FAQ
What's the difference between mutually exclusive and independent?
Mutually exclusive means the two events cannot both happen, so P(A ∩ B) = 0. Independent means one happening tells you nothing about the other, so P(A ∩ B) = P(A)P(B). They are almost opposites: two events with positive probability that are mutually exclusive are necessarily dependent, because if one occurs the other definitely does not. Mixing these up is a classic exam slip.
How do I set up a Bayes' theorem question?
Name the events and write down every probability you are given, including the conditionals. Build the denominator with the law of total probability — usually P(+) = P(+|D)P(D) + P(+|not D)P(not D). Then apply P(D|+) = P(+|D)P(D)/P(+). Laying out the events and the total-probability denominator explicitly is where the method marks are.
Why is the answer to the disease test so low?
Because of the base-rate trap. When a condition is rare, even an accurate test produces many false positives from the large healthy majority, and these can outnumber the true positives. Bayes' theorem makes this precise: the prior P(D) sits in both the numerator and the denominator, so a small prior pulls the posterior P(D|+) down. It is the single most-tested intuition in the probability block.
Do I need to memorise the distribution formulas?
No — the means and variances of the named discrete and continuous distributions are on the supplied formula sheet. What you must do by hand is recognise which distribution a scenario fits (binomial, Poisson, uniform, normal and so on), then apply its formula and compute the exact value. So practise identifying the distribution, not memorising its moments.
Exam move
Bayes' theorem is the signature long-answer, so drill the full chain until it is automatic: name the events → build P(B) with the law of total probability → apply Bayes. Always write the total-probability denominator out in full — it carries marks and prevents arithmetic slips. Keep the base-rate intuition sharp: a rare condition plus an imperfect test gives a low posterior, and examiners love that surprise. Be precise about mutually exclusive (cannot co-occur) versus independent (no information transfer), since they are tested as a deliberate trap. For random variables, the move is E[X] = Σ x·P(X = x); for distribution questions, identify the named distribution first, then pull its mean and variance off the formula sheet. Keep answers exact — fractions in lowest terms unless a decimal is asked.
