The AskSia course library carries 35 hypothesis-testing study guides across 20 courses at 8 universities. The procedure inside them is identical. The chapter title almost never is.
Those first two numbers come from a study that handed six statements about a single test result to psychologists at German universities. Every statement was false. Almost nobody caught them all.
That is the honest starting point for this topic. The mechanics are learnable in an afternoon. The interpretation defeats professionals.
What Is Hypothesis Testing?
Hypothesis testing is a decision procedure. You assume a specific claim about a population is true, calculate how unusual your sample would be under that assumption, and reject the claim if the sample is unusual enough.
The assumed claim is the null hypothesis, written H0. The rival claim is the alternative, H1 or Ha.
A concrete case. A campus cafe claims mean wait time is 4 minutes. You time 36 orders and get a sample mean of 4.6 minutes with a standard deviation of 1.5. The one-sample t-statistic is 2.4, and with 35 degrees of freedom the two-sided p-value lands near 0.022.
At a 5% threshold you reject the 4-minute claim. At a 1% threshold you do not. Same data, different verdict.
That sensitivity to a threshold you chose yourself is not a flaw in your arithmetic. It is the design.
What Are the Five Steps?
Almost every course compresses the procedure into five moves. The wording shifts between textbooks. The sequence does not.
- State H0 and H1. H0 always contains the equality. H1 carries the direction you suspect, one-sided or two-sided. Pick this before you look at the data.
- Set the significance level. Conventionally α = 0.05, sometimes 0.01 in medicine or 0.10 in exploratory economics. This is the error rate you accept, not a property of the data.
- Choose the test and check its conditions. Independence, sample size, distribution shape, equal variances. Marks are lost here more often than anywhere else.
- Compute the test statistic and p-value. The statistic measures distance from H0 in standard error units. The p-value converts that distance into a probability.
- Decide and state the conclusion in context. If p ≤ α, reject H0. Otherwise fail to reject. Write the verdict in the language of the original problem, not in symbols.
Step 3 is where most exam papers actually separate students. The scenario names a research question, never a test.
Mock Exam mode is built for this specific failure. It presents the scenario in exam phrasing and grades whether you selected the right test before it grades your arithmetic.
What Do H0 and H1 Claim?
H0 is the claim of no effect, no difference, no relationship. It is the position you hold until the evidence forces you off it.
H1 is everything else. It is deliberately vague about magnitude.
The asymmetry matters more than most first-year courses admit. You can reject H0. You can never accept it. A p-value of 0.62 means your data are compatible with no effect and also compatible with a small effect you lacked the sample size to detect.
Writing "we accept the null hypothesis" is the single most common wording penalty in introductory statistics marking. The correct phrase is "fail to reject."
The confidence interval for the same parameter carries strictly more information than the test result. A 95% interval of −0.4 to 8.9 tells you the effect might be nothing, or might be large. The p-value tells you only that it crossed a line.
Why Do the Two Error Types Matter?
Two ways to be wrong, and they trade against each other. Tighten one and you loosen the other.
Power is 1 − β, and it depends on sample size, effect size, and α together. Doubling your sample is usually the only lever a student controls.
Exam questions rarely ask you to define these terms. They ask you to identify which error a described scenario represents, then say who bears the cost.
A drug regulator fears Type I errors: approving something inert. A screening program fears Type II errors: missing disease. The α you choose encodes which mistake you consider worse.
Where Does Your Course Teach This?
Search your syllabus for "hypothesis testing" and you may find nothing, even though your course teaches it in week 7.
The naming difference is not cosmetic. MAST90105 derives the Neyman-Pearson lemma and asks you to prove a test is most powerful. ECMT1010 builds the null distribution by shuffling data and never mentions a t-table.
Both are hypothesis testing. Studying the wrong one costs you the exam.
Why Do Experts Misread p-Values?
Because the correct definition is counterintuitive and every shorter version of it is wrong.
A p-value is the probability of observing data at least as extreme as yours, assuming H0 is true. It is a statement about data given a hypothesis. It is not a statement about a hypothesis given data.
Haller and Krauss tested this in 2002, publishing in Methods of Psychological Research Online. They gave psychologists a result of t = 2.7, df = 18, p = 0.01, then six statements about what it meant. All six were false.
Of 30 methodology instructors who taught statistics, 6 marked all six as false. Of 44 students, none did. The instructors averaged 1.9 wrongly endorsed statements; the students averaged 2.5.
The discomfort this created has reached the discipline's leadership. In March 2019, Amrhein, Greenland and McShane published a comment in Nature volume 567, pages 305 to 307, signed by more than 800 researchers, arguing that the significant and non-significant labels should be retired.
They did not call for banning p-values. They called for reporting intervals and effect sizes instead of a binary verdict.
These six statements are pure recall content under time pressure, which is why they suit spaced repetition. AskSia's Flashcards with FSRS scheduling will surface them at widening intervals tuned to your exam date rather than the night before.
Frequently Asked Questions
What is hypothesis testing and examples?
Hypothesis testing is a procedure for deciding whether sample evidence is strong enough to reject a stated claim about a population. You assume the claim, called H0, then measure how unusual your sample would be if it held. Three worked examples: a cafe claims a 4-minute mean wait, you time 36 orders, get a sample mean of 4.6 with a t-statistic of 2.4 and p near 0.022, and reject at α = 0.05. A marketing team compares click rates of 10% and 8% between two ads and runs a two-proportion z-test. An econometrician tests whether 3 regression coefficients are jointly zero using an F-test, the standard ECON2515 and ETF2100 exam question. The procedure is identical across all three. Only the statistic changes. Work through a full example in the Stats 101 cheatsheet before attempting past papers.
What are the 5 steps of hypothesis testing?
State H0 and H1. Set the significance level α, conventionally 0.05, sometimes 0.01 in clinical work. Choose the test and verify its conditions, including independence, sample size and distribution shape. Compute the test statistic and p-value. Decide and state the conclusion in the context of the original question. Two details decide marks. First, H0 and α must be fixed before you inspect the data, because choosing α after seeing p invalidates the error rate you claim to control. Second, step 3 carries the most marks in most rubrics and is the step scenarios deliberately obscure, since exam questions describe a research situation and never name the test. If your course is one of the 20 in the AskSia library, check your chapter guide for the notation convention your marker expects.
What is H0 and H1 in a hypothesis?
H0, the null hypothesis, states no effect, no difference, or no relationship, and always contains the equality sign. H1, the alternative, states what you suspect, and may be one-sided or two-sided. The asymmetry between them is the part courses under-teach. You can reject H0 when evidence is strong. You can never accept it, because failing to reject is compatible with both a true null and a real effect too small for your sample to detect. Writing "we accept the null hypothesis" is among the most common wording penalties in introductory statistics marking, and the correct phrase is "fail to reject H0." A 95% confidence interval running from −0.4 to 8.9 makes the ambiguity visible in a way p = 0.11 never does. Report the interval alongside the test.
Is hypothesis testing difficult?
The mechanics are not. The interpretation is, and there is evidence for that claim rather than reassurance. Haller and Krauss found in 2002 that among 30 university methodology instructors who teach statistics, only 6 correctly identified all six false readings of a single p-value. None of 44 students managed it. Instructors wrongly endorsed 1.9 statements on average, students 2.5. In 2019, more than 800 researchers signed a Nature comment arguing the significance label should be retired outright. So difficulty here is not a personal failing. It is a known property of the concept. Practically, this means you should over-invest in the definition of the p-value and the two error types, and under-invest in memorising formulas your exam gives you. AskSia's AI tutor will re-explain the conditional structure three different ways until one version lands.
What is a hypothesis and how is it tested?
A statistical hypothesis is a precise claim about a population parameter, such as a mean, proportion or regression coefficient. It is not a general research idea. "Sleep affects grades" is a research question; "the mean GPA difference between the two sleep groups is zero" is a hypothesis you can test. Testing converts the claim into a probability statement. You calculate a statistic measuring the distance between your sample and the claimed value in standard error units, then convert that distance into a p-value. Compare the p-value to α and decide. The conversion step depends entirely on which test fits your data shape, which is why 6 question types cover most undergraduate marks. Map your own unit's chapter against the six shapes using the PSYC10004 NHST guide if your course uses psychology notation.
Which hypothesis test should you use?
Match the test to the shape of the question, not the wording of the topic. One mean against a stated value takes a one-sample t-test. Two group means take a two-sample t-test. A single proportion takes a z-test. Two categorical variables take a chi-square test of independence. Three or more means take a one-way ANOVA. Several regression coefficients jointly zero take an F-test. That covers 6 shapes and the large majority of undergraduate test-selection marks. Then check conditions before committing, because independence violations and small unequal-variance samples change the correct answer. Econometrics courses add a seventh layer, where heteroskedasticity-robust standard errors change the test statistic without changing the hypothesis. The econometrics workspace handles that distinction directly. Practise selection separately from computation, because exams test them separately.
When Is Hypothesis Testing the Wrong Tool?
When the question is how large, not whether. Testing answers a yes-or-no question about a threshold. If your actual interest is the size of an effect and its uncertainty, an interval estimate answers it directly and the test adds nothing.
When you are exploring rather than confirming. Running 20 tests at α = 0.05 on genuinely null data produces roughly 1 significant result by construction.
When the sample is large enough that trivially small differences clear the threshold. With 500,000 observations, a 0.02% difference can return p below 0.001 while being commercially and scientifically meaningless.
The 2019 Nature signatories were pointing at exactly this. The procedure is sound. The binary label attached to its output carries more weight than the mathematics can support.
Your course will still examine it in the classical form. Learn it that way, and hold the caveats separately.
