SOAD2021 Chap.4 Program Evaluation and Theory of Change
Program Evaluation and Theory of Change
Program evaluation asks systematic questions about implementation, outputs, outcomes and value. A theory of change explains why activities are expected to contribute to change, while a program logic offers a concise sequence of inputs, activities, outputs and outcomes. Neither is credible when arrows replace unstated assumptions.
Build the causal story backward from a desired outcome.
Identify the behaviour, service or environmental change required; the mechanisms that could produce it; the activities intended to activate those mechanisms; and the contextual conditions that must hold. Then test each link with a counter-question: what else could produce this result, and what would show the link did not operate?
Indicators translate parts of the logic into observable information.
Every indicator needs a definition, data source, frequency, responsibility and intended use. Balance implementation measures with outcome measures so a disappointing result can be investigated rather than merely reported. Risks and assumptions belong in the design because they determine what evidence must be gathered and when adaptation is warranted.
Evaluation questions should be mapped to users and decision points.
A question about reach may inform recruitment changes during delivery, while a question about sustained outcomes may require a later review and a credible comparison. Write the intended use beside each question, then remove any measure that cannot inform that use. This exposes vanity indicators and prevents a dashboard from replacing judgement.
When results are mixed, trace them through the causal chain: strong activity counts with weak intermediate outcomes may indicate a missing mechanism, poor measurement or a contextual barrier rather than simple program failure. Seek evidence capable of distinguishing those explanations. The final judgement can therefore recommend continuation, adaptation, scale or closure with explicit conditions.
A MERL plan is strongest when it specifies not only what will be measured, but who will review the evidence, how competing interpretations will be considered and which signal triggers action.
What this chapter covers
- 01
Program evaluation
- 02
Implementation science
- 03
Theory of change
- 04
Program logic
- 05
Indicators
- 06
Assumptions
- 07
Risk and learning
Worked example · free
Worked example: repair a weak theory of change
- 1Name the immediate output, such as staff completing training and practising a defined skill.
- 1Add the mechanism and intermediate change, such as more consistent engagement leading to timely support plans.
- 1State assumptions about supervision, workload, service availability and family choice.
- 1Attach implementation and outcome indicators that could distinguish non-delivery from theory failure.
Key terms
- Implementation fidelity
- Implementation fidelity is the extent to which a program is delivered as intended across specified core components. Adaptation can be recorded separately rather than automatically treated as failure.
- Intermediate outcome
- An intermediate outcome is a change expected between immediate outputs and longer-term goals. It helps test whether a proposed mechanism is operating.
- Evaluability
- Evaluability is the extent to which a program has sufficient clarity, evidence access and stakeholder agreement for useful evaluation. It can be improved before a full evaluation begins.
Program Evaluation and Theory of Change FAQ
What is the difference between program logic and theory of change?
A program logic is usually a compact representation of resources, actions and results. A theory of change explains the mechanisms, assumptions and context that make those links plausible. They support each other, but a sequence of boxes alone is not a complete causal explanation.
How many indicators should a plan include?
Use the smallest set that can inform the intended decisions and diagnose important links. More indicators increase cost, burden and interpretive complexity. Prioritise measures with clear definitions, feasible sources, relevant timing and an identified person who will use the result.
What does a poor outcome result tell me?
On its own, it shows that the desired result was not observed under the chosen measure and period. Implementation evidence can help distinguish incomplete delivery, weak reach, contextual disruption, unsuitable measurement and a faulty causal theory. The response should follow that diagnosis.
Where should assumptions appear?
Place assumptions beside the link they support and pair important assumptions with a monitoring signal where possible. A separate list can hide their consequences. Ask who controls the condition, how likely it is to change and what adaptation becomes necessary if it fails.
Assessment move
Draw a program logic from memory, then annotate every arrow with a mechanism and every box with a possible indicator. Use two colours to separate implementation from outcome evidence. Create one adverse scenario, such as low reach or staff turnover, and trace which assumptions fail and which measurements would reveal the problem.
Finish by explaining the model to a fictional decision maker in one minute, including the strongest uncertainty and proposed learning response. Then remove one indicator and defend whether the remaining evidence can still diagnose implementation failure separately from theory failure. Set a review signal for the most fragile assumption and name the adaptation that signal would trigger.
