CMNS2600 Chap.3 Audience Research Instruments
Audience Research Instruments
Research instruments produce different forms of audience evidence. Interviews elicit reflective accounts, ethnography follows practice in context, surveys standardise questions and digital traces record actions selected by a system. None is automatically deep, broad or objective. This chapter focuses on question fit, sampling, wording, observation, inference and triangulation.
The strongest method section explains how evidence was generated, who was included, which actions or meanings remain invisible and why the conclusion is no broader than the population, setting and period observed. Instrument design also shapes the response before analysis begins. A prompt defines what can be said, a sampling route determines who can appear and a platform log inherits the categories built into the service.
Comparing methods is therefore a comparison of production conditions, not simply of data formats. Triangulation earns its value when different sources can expose one another's omissions.
What this chapter covers
- 01
Instrument choice through inferential need
- 02
Reported accounts compared with observed behaviour
- 03
Semi-structured interviews and responsive probing
- 04
Ethnographic attention to practice and setting
- 05
Survey wording, reference periods and response options
- 06
Sampling frames, coverage and non-response
- 07
Digital traces as platform-selected records
- 08
Triangulation through comparison and divergence
Worked example · free
Design an interview sequence
- +1Open with the last episode chosen and why play was pressed.
- +1Ask what made the episode credible or doubtful.
- +1Probe a specific moment and its context.
- +1Seek a counter-example where the same host was trusted less.
Key terms
- Semi-Structured Interview
- A guided conversation using common topics with flexible probes for participant meaning and examples.
- Ethnography
- Sustained inquiry into practice, meaning and relation within everyday settings.
- Sampling Frame
- The practical list or route through which members of the target population can be recruited.
- Coverage Error
- Bias created when the sampling route omits part of the population the claim concerns.
- Measurement Error
- Distortion introduced when wording, format or procedure does not capture the intended construct.
- Triangulation
- Comparison among methods, sources or perspectives to examine convergence, divergence and blind spots.
Audience Research Instruments FAQ
Should a project use qualitative or quantitative methods?
Choose according to the evidence the research question requires. A question about meaning may need interviews or observation; a question about distribution may need standardised measures and a suitable sample. Many projects combine methods, but each instrument should have a defined role rather than being added for appearance.
What makes an interview question weak?
Leading language, double questions, unexplained theory terms and vague prompts all reduce usefulness. Strong questions invite description, ask one thing at a time and make room for disagreement. Follow abstract answers with probes for examples, sequence and context instead of moving immediately to the next scripted item.
Why can a large survey still mislead?
Sample size cannot repair a poor sampling frame, non-response or biased wording. A large convenience sample may estimate the wrong population very precisely. Evaluate who could be reached, who answered, how the question defined the behaviour and whether the conclusion stays within those boundaries.
Assessment move
Build an instrument table with columns for evidence produced, best use, main distortion and claim limit. Rewrite one research question for interviews, ethnography, surveys and traces; notice how the question changes when the evidence changes. Practise repairing leading and double-barrelled questions. For sampling, write the target population and actual recruitment route on separate lines. If they differ, name the gap.
When combining methods, write the purpose of each before describing procedure. Treat disagreement among methods as a finding to explain, not noise to erase. Create a small instrument workshop for one neutral topic. Draft an interview opening, two probes, a survey item with balanced response options, an observation note template and a definition of the trace you would count.
After each item, state the inference it permits and one plausible misinterpretation. This makes wording and measurement part of the analysis rather than administrative detail. Pilot every question by answering it from three imagined positions, including a participant who rejects its premise. Look for undefined time periods, overlapping categories, moral language and requests that contain two judgements.
Revise until a respondent can give an accurate answer that the researcher did not expect. For sampling, draw a funnel from target population to accessible population, invitation, participation and retained cases. At every narrowing point, ask which audience member is more likely to disappear. Finish by designing triangulation that can genuinely challenge a result.
Pair a trace of repeated playback with an interview about motive, or survey prevalence with observation of practice. Predict one way the sources might disagree and write how that disagreement would change the conclusion. A robust mixed-method plan does not use the second source as decoration; it gives the first source a real opportunity to be corrected.
