EDF1401 Chap.10 Evaluation and Continuous Improvement
Evaluation and Continuous Improvement
Evaluation judges the effectiveness of a learning design and informs improvement. It should be planned during analysis, not attached after launch. Formative evaluation improves a design while changes can still be made; summative evaluation supports an overall judgement after implementation.
Kirkpatrick's four levels prompt questions about reaction, learning, behaviour and results, but higher numbers are not automatic proof of better evidence. Each level requires a defensible link to objectives, context and timing, and broader results may be influenced by many factors.
What this chapter covers
- 01
Evaluation for judgement and continuous improvement
- 02
Formative and summative evaluation timing
- 03
Kirkpatrick Level 1: reaction
- 04
Kirkpatrick Level 2: learning
- 05
Kirkpatrick Level 3: behaviour or transfer
- 06
Kirkpatrick Level 4: wider results and attribution
Worked example · free
Build evidence beyond satisfaction
- +2Level 1 shows positive reaction but does not establish knowledge, skill, transfer or organisational results.
- +2Add Level 2 evidence through an aligned scenario or demonstration, ideally compared with a baseline.
- +2Add Level 3 observation of workplace behaviour after sufficient opportunity, including barriers and supervisory conditions.
- +2Interpret Level 4 incident data cautiously because frequency, reporting and environmental factors complicate attribution.
Key terms
- Formative evaluation
- Evaluation used during design or operation to guide improvement while change is possible.
- Summative evaluation
- Evaluation used after implementation to support an overall judgement of effectiveness.
- Reaction evidence
- Information about participant perceptions, relevance, engagement or satisfaction.
- Learning evidence
- Information about changes in intended knowledge, skill, attitude or confidence.
- Behaviour evidence
- Information about whether intended performance occurs in the relevant real-world context.
- Results evidence
- Information about wider organisational or social outcomes connected with a learning intervention.
Evaluation and Continuous Improvement FAQ
Is a satisfaction survey enough to evaluate learning?
No. It can provide Level 1 reaction evidence, which may help improve relevance or experience, but it does not directly show learning, transfer or results. Match evidence to the evaluation question. If the goal is performance in context, include an aligned performance measure and later behaviour evidence.
Does evaluation happen only after implementation?
No. Evaluation should inform analysis, prototypes, resources and implementation as well as later judgement. Formative evaluation catches problems while change is possible, and baseline planning begins early. Summative evaluation occurs after implementation, but even then its findings can guide the next design cycle and continuous improvement.
Is Kirkpatrick Level 4 always the best level?
No. The appropriate level depends on the objective and decision. Wider results can be valuable but are often influenced by policies, resources, opportunities and other interventions, making attribution difficult. A valid Level 2 or Level 3 measure may answer the design question more directly than a weak Level 4 indicator.
Why does evaluation need a baseline?
A baseline helps distinguish current performance from later change and reveals starting variation. It does not by itself prove that the design caused a change, but it strengthens interpretation and planning. Choose a baseline aligned with the intended outcome and collect it early enough to inform design or later comparison.
What should I do when evaluation evidence is mixed?
Report what each measure supports and avoid collapsing unlike evidence into one success score. Positive reaction with weak transfer suggests a different problem from weak learning with strong opportunity. Investigate competing explanations, identify the design decision affected and specify which further evidence would distinguish them.
Assessment move
Build an evaluation staircase for one learning design. At each level, write the question, timing, measure, decision and major limitation. For reaction, avoid treating satisfaction as learning. For learning, align evidence with the objective and gather a baseline where useful. For behaviour, allow time and opportunity for transfer and record workplace supports or barriers.
For results, identify other factors that could influence the outcome and avoid causal claims the design cannot support. Add formative checks at analysis, prototype, development and implementation stages so evaluation is not reserved for the end. Practise with incomplete evidence, such as high satisfaction but unchanged performance, and state exactly what can and cannot be concluded.
In portfolio commentary, connect an evaluation measure to a design decision and explain how findings would trigger revision. Use a compact claim-evidence-caveat format: claim only at the level supported, name the evidence, then acknowledge timing or attribution limits.
End with a specific improvement action, because evaluation in this unit serves continuous improvement rather than merely classifying a project as successful or unsuccessful.
Working through Evaluation and Continuous Improvement in EDF1401? Sia is AskSia’s AI Education tutor — ask any EDF1401 Evaluation and Continuous Improvement question and get a clear, step-by-step explanation grounded in how EDF1401 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
