APG5452 Chap.4 Evaluation, Impact and Evidence
Evaluation, Impact and Evidence
Evaluation judges how well a CEEF fulfilled its purposes, while impact concerns the change associated with it, the affected group, timescale and basis for attribution. This chapter builds an evidence chain from purpose through indicator and method to finding and warranted claim.
It separates delivery outputs from experiences, short-term outcomes and durable impacts, then considers how value is distributed among organisers, funders, artists, workers, participants and affected publics. The aim is to make every conclusion proportional to what the method can actually observe and to turn evaluation into a useful design process rather than an end-of-project decoration.
What this chapter covers
- 01
Purpose and evaluation questions
- 02
Outputs, outcomes and impacts
- 03
Indicators and methods
- 04
Triangulation
- 05
Attribution and timing
- 06
Distribution of value
Worked example · free
Evaluate an intergenerational exhibition
- +1Define exchange through observable listening, dialogue and revision rather than a broad transformation claim.
- +2Observe workshops and compare early and final audio-label drafts.
- +2Interview participants from both age groups and review curatorial decisions.
- +1Report documented exchange while acknowledging continued institutional control.
Key terms
- Indicator
- An observable sign selected to represent progress toward a defined purpose or outcome.
- Outcome
- A change occurring after activity, such as knowledge, confidence, connection, intention or practice.
- Attribution
- Reasoning about whether and how an observed change can be linked to the CEEF rather than other causes.
- Triangulation
- The use of different evidence sources or methods to compare findings and reduce reliance on one blind spot.
Evaluation, Impact and Evidence FAQ
What is the difference between evaluation and impact?
Evaluation judges performance against stated purposes and can include design, delivery, experience and outcomes. Impact refers more specifically to change associated with the CEEF. An impact claim should name what changed, for whom, over what period and how the evidence links the activity to that change.
Why are attendance and reach not enough?
They document scale of contact or circulation, which may be an important output. They do not show what people experienced, learned or influenced, nor whether a change lasted. Match those larger questions with observation, interviews, comparison, follow-up and evidence from differently positioned constituencies.
How can a proposal evaluate inclusion?
Define inclusion through concrete relations such as access, participation, decision power, payment, safety or continuity. Then select evidence for those relations and include people who did not attend or who carried costs. Representation and satisfaction can contribute evidence, but neither alone demonstrates institutional inclusion.
How do outputs differ from outcomes?
Outputs describe what was delivered, such as sessions, commissions, attendance or access provisions. Outcomes concern changes that follow, such as knowledge, confidence, connection or practice. Link them carefully because delivering more activity does not establish that the intended change occurred.
What does triangulation add to evaluation?
Triangulation compares evidence from methods with different strengths and blind spots. Observation may reveal behaviour, interviews interpretation and documents institutional decisions. Agreement can strengthen a finding, while contradiction can expose uneven experience or a gap between intention and practice that deserves analysis.
When is a transformation claim justified?
The study must define the change, establish a starting point, identify the affected group, track an appropriate timescale and reason about the CEEF's contribution. Without those elements, describe the documented experience or short-term outcome rather than expanding it into a durable transformation claim.
How should contradictory findings be reported?
Treat disagreement as information rather than averaging it away. Identify the roles and conditions associated with different accounts, test possible explanations and narrow the conclusion. Contradiction may reveal uneven experience, competing purposes or a method that captures only one part of the CEEF.
Assessment move
For every value or impact claim, draw five boxes labelled purpose, indicator, method, finding and conclusion. Fill them in without skipping a box. If the purpose is community connection, attendance is only an output; choose evidence about interaction, continued relation or participant interpretation.
Practise matching methods to questions: observation for behaviour and context, interviews for reasoning, surveys for reported patterns, documents for intention and decisions, and longitudinal comparison for persistence. Record limitations beside each method before analysing results. Separate immediate experience, short-term outcome and durable impact so enthusiasm is not mistaken for lasting change.
Include distribution by asking who gained, who carried costs and whose definition of value shaped the indicators. When writing, use verbs that fit the evidence: documents, indicates, suggests, illuminates or supports. Reserve proves and transforms for designs capable of sustaining that burden.
In a proposal, state how evaluation findings would change a future decision; this turns measurement into learning rather than ceremonial reporting. Draft a negative-result paragraph before collecting data. State what it would mean if the indicator did not move, who would need to know and which design decision could change. This prepares you to learn rather than defend the project.
Audit incentives as well: a reach target can push teams toward volume, while a satisfaction target may suppress difficult but valuable encounters. Balance measures and explain the trade-off. Finally, disaggregate findings wherever the purpose concerns access or equity, because an average can conceal sharply different experiences among constituencies.
