AGRI90093 Chap.9 Evaluation Methods and Evidence
Evaluation Methods and Evidence
Method selection follows the evaluation question, population, timing, ethics and intended use. This chapter compares surveys, interviews, observation, routine records, workshops, narratives and case studies by breadth, depth and resource demands. Mixed methods become coherent when each source answers a distinct part of the key question and disagreement is investigated rather than treated as error.
What this chapter covers
- 01
Statistics, surveys and records for breadth, consistency and repeated evidence
- 02
Interviews, narratives and case studies for pathways, context and contrasting experience
- 03
Observation and workshops for practice, interaction and collective interpretation
- 04
Sampling, wording, consent, confidentiality and researcher influence
- 05
Complementary mixed methods and analysis of divergent findings
- 06
Explicit boundaries, intended use and decisions linked to evidence
Worked example · free
Evaluate a regional advisory pilot
- 1Use service logs to describe reach, timing and repeated contact without treating contact as an outcome.
- 1Use a short survey for comparable reports of usefulness and intended practice.
- 1Interview contrasting users to explain high, low and discontinued use while protecting confidentiality.
- 1Integrate findings around one key question and link barriers to a delivery redesign.
Key terms
- Mixed Methods
- A design that combines complementary forms of evidence to answer different parts of an inquiry.
- Purposive Sampling
- Selection of cases because their characteristics or experiences illuminate the evaluation question.
- Triangulation
- Comparison of evidence sources to examine convergence, divergence and different aspects of a claim.
- Negative Case
- A case that does not fit the main pattern and can test or refine an explanation.
Evaluation Methods and Evidence FAQ
When should I use a survey?
Use a survey when consistent questions across a broader group can answer part of the evaluation inquiry. Keep it proportionate, attend to sampling and wording, and do not ask fixed items to explain every mechanism. Interviews or observation may be needed for depth.
What does triangulation add to an evaluation?
Triangulation compares sources that contribute different evidence. A survey can show how common a report is, interviews can explain pathways and records can check repeated behaviour. Disagreement is informative because it may reveal measurement limits, subgroup differences or a weak assumption.
Why are attendance records insufficient for outcome claims?
Attendance records establish reach or participation, not learning, practice change or higher effects. Outcome claims need evidence at the relevant level, such as demonstrated capacity, observed or documented use and contextual measures tied to the program logic.
Assessment move
Build a method table with the columns question contribution, breadth, depth, burden, ethical issue, limitation and intended use. Practise selecting two or three complementary sources under realistic constraints. Explain why each method belongs rather than listing tools. Include sampling and confidentiality where the topic is sensitive. When sources conflict, generate possible explanations and identify further checks.
End the plan with a comparison strategy and decision rule so data collection remains connected to program improvement. Select methods only after writing the evaluation question and clarifying the claim it requires.
A question about reach needs evidence on who participated and who did not; a question about mechanism needs evidence about experiences and processes; a question about practice change needs observation, records or credible accounts linked to action. Write one sentence explaining the contribution of each source. Remove any method that repeats another source without adding coverage, depth, comparison or corroboration.
Develop a sampling rationale suited to the question. Define the relevant population or cases, identify whose experience may be missed and explain whether the priority is representation, variation, depth or access to a particular process. Convenience may be unavoidable, but name the resulting limits. For qualitative work, select participants or cases that can illuminate meaningful contrasts.
For quantitative work, check whether the available numbers and response patterns support the intended comparison. Add an ethics and burden column before finalising the plan. Consider consent, confidentiality, sensitive topics, power relationships, data ownership and the practical demands placed on participants and staff. Explain how data will be stored, reported and separated where identification could cause harm.
Reduce collection that has no clear use. In small communities, removing names may not be enough to prevent recognition, so reporting choices need contextual judgement. Plan triangulation as an inquiry into convergence and difference. State what each source can and cannot show, then specify how evidence will be compared.
Agreement may strengthen confidence, while disagreement can reveal uneven implementation, different experiences, timing effects or measurement problems. Do not resolve conflict by automatically privileging the numerical source. Generate explanations and identify a follow-up check that could distinguish among them. Turn analysis into a decision protocol.
Define thresholds, patterns or qualitative findings that would prompt continuation, redesign or further inquiry, while preserving room for stakeholder interpretation. Identify who participates in sense-making and how conclusions will be communicated to different audiences.
End by matching every conclusion to the strength of the evidence: distinguish description, association, participant explanation and causal attribution so the report does not outrun the design.
Working through Evaluation Methods and Evidence in AGRI90093? Sia is AskSia’s AI Education tutor — ask any AGRI90093 Evaluation Methods and Evidence question and get a clear, step-by-step explanation grounded in how AGRI90093 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
