POPH90068 Chap.10 Program Evaluation
Program Evaluation
Evaluation tests the links between resources, delivery, reach, behaviour or service change and population outcomes. Process measures explain whether a program was delivered as intended and why a pathway held or failed. Outcome measures require a clear denominator, time window, comparison and data source.
A feasible plan begins with the decision that results will inform and selects a proximal primary outcome the intervention can plausibly influence. Linked indicators allow a disappointing endpoint to be traced back to delivery, reach or mechanism instead of being labelled simply as failure.
Ethics, data governance and distributional analysis are treated as design requirements because the same measure can inform improvement or create avoidable harm.
What this chapter covers
- 01
Logic models
- 02
Fidelity and dose
- 03
Reach and acceptability
- 04
Behaviour and service outcomes
- 05
Epidemiological outcomes
- 06
Comparison, timing and decisions
Worked example · free
Evaluate extended testing hours
- 1Define completed testing in the intended population as the primary outcome.
- 1Use eligible bookings as a denominator and compare with a suitable prior or concurrent service.
- 1Track cancellations, delay, positivity, satisfaction and delivery disruptions.
- 1Predefine what result will trigger continuation or redesign.
Key terms
- Logic model
- A representation linking resources, activities, outputs and expected outcomes.
- Fidelity
- The extent to which delivery followed the intended program design.
- Reach
- The proportion and characteristics of the intended population that participated.
- Process measure
- An indicator of implementation, delivery, uptake or intermediate pathway performance.
- Outcome measure
- An indicator of the change the program ultimately seeks to influence.
Program Evaluation FAQ
What is the difference between process and outcome evaluation?
Process evaluation examines delivery, fidelity, reach, acceptability and barriers. Outcome evaluation examines changes in behaviour, services or health. Both are needed to explain whether the intervention pathway operated.
Why can notifications rise after a successful testing program?
Greater testing can detect infections that were previously missed, increasing notifications even while access improves. Interpret testing coverage, positivity and new-infection measures alongside the program change.
What makes an evaluation feasible?
The indicator is collectable within the time and resources available, uses a defined denominator and comparison, protects participants and can inform a real decision. A distant population endpoint may be unsuitable for a short local program.
Assessment move
Work backwards from the decision. State what the team needs to decide, then choose the outcome that can inform it. Define numerator, denominator, population and time window before naming a data source. Add one implementation measure that could explain the result and one equity measure that could expose uneven reach.
Practise distinguishing a null effect caused by weak delivery from a well-delivered intervention that did not change the outcome. In proposal review, reject any objective whose evaluation would require unavailable data or an implausible time horizon. Construct a logic chain before choosing a favourite metric.
Write inputs, delivery, reach, immediate response, service or behavioural outcome and longer-term health outcome in sequence. Above every arrow, state the assumption that must hold. Then choose at least one measure near the activity and one near the intended result.
If the endpoint does not move, these linked measures help distinguish absent delivery, poor reach, a failed mechanism and an outcome that simply requires more time. Specify indicators as if another evaluator must calculate them. Name the numerator, denominator, eligible population, setting, observation period and data source. Define repeat events and missing data where they could change interpretation.
Avoid verbs such as improve or increase without a baseline and comparison. For qualitative outcomes, state whose experience is sought, how participants are selected and which decision the findings will inform. Precision in definitions exposes objectives that sound measurable but are not operational. Use a data-source comparison exercise.
For each question, consider routine notifications, service records, surveys and interviews. Record what each source observes, whom it misses, how quickly it becomes available and what bias or governance concern matters. A source with broad coverage may lack detail; a detailed service cohort may not represent the population; a survey may reveal behaviour while facing selection and response limits.
Select the source that fits the decision instead of claiming that one dataset is universally strongest. Practise interpreting discordant findings. If delivery is high but reach is low, examine recruitment and access. If reach is high but the immediate outcome is unchanged, reconsider the mechanism or intervention quality.
If proximal outcomes improve while notifications rise, assess testing and detection before declaring failure. If averages improve but a subgroup does not, treat distribution as a result requiring action. In each case, write the next operational decision rather than stopping at a statistical description. Close with feasibility and ethics.
Ask whether the team can obtain the data within the proposed period, whether collection is proportionate to the decision, how sensitive information is protected and who has authority to access or interpret it. Predefine thresholds for continue, adapt or stop, while allowing contextual evidence to explain unexpected results.
An evaluation plan is complete when every measure has a purpose, every inference has a boundary and findings can lead to a credible program decision.
