EDS6003 Chap.8 Survey Instruments and Administration
Survey Instruments and Administration
Scores are built through translations
Assignment 3 asks students to identify an appropriate questionnaire of about twenty items, excluding demographic information, and use a three-to-five-point scale. The construct, wording, response options and administration all shape what the final score can mean.
A correct calculation cannot repair an item that mixes two ideas or supplies the preferred answer.
Administration changes the evidence
Invitation source, privacy, timing, reminders and access influence who responds and what they disclose. Piloting should therefore test interpretation as well as completion time.
Cognitive probing asks respondents what they understood, how they selected an option and whether an important response was unavailable. Reports should describe achieved participants, missing responses and recruitment conditions rather than only naming the intended audience.
What this chapter covers
- 01
Constructs and operationalisation
- 02
Single-purpose item wording
- 03
Response scales
- 04
Leading and double-barrelled items
- 05
Cognitive piloting
- 06
Recruitment and administration bias
Worked example · free
Repair an item that manufactures agreement
- 3Identify leading and double-barrelled wording.
- 3Write separate behavioural items.
- 2Add a useful timeframe.
Key terms
- Construct
- An abstract idea such as confidence or belonging that must be represented through observable measures.
- Operationalisation
- The process of translating a construct into items, observations or scores that can be analysed.
- Response Scale
- The ordered set of answers a respondent may choose for a survey item.
- Double-barrelled Item
- A question combining two claims that may receive different answers.
- Cognitive Interview
- A pilot technique examining how respondents understand an item and select an answer.
Survey Instruments and Administration FAQ
Why should survey items ask one thing at a time?
When an item combines two ideas, one response cannot show which part the respondent accepted. Splitting the item preserves interpretability and prevents a convenient score from hiding disagreement between the underlying constructs.
How should response options be chosen?
Match labels to the question being asked, such as agreement, frequency or confidence, and cover plausible positions including not applicable where needed. Ordered labels do not automatically create equal numerical intervals, so analysis should respect what the scale can support.
What must be reported about survey administration?
Describe recruitment route, access, timing, consent, reminders, exclusions and actual respondents. A link intended for a whole cohort may still produce a selective volunteer sample. Administration details show how the observed data came to exist and what claims remain defensible.
What should be recorded during survey administration?
Keep counts for invitations, starts, completions, exclusions and reminders, together with the exact questionnaire version and access period. Examine item-level missingness and response by relevant subgroup before dropping incomplete cases. Preserve the relationship between participant-facing labels and exported variables.
These records show how the achieved dataset was produced and whether administration or nonresponse changed the group the final summaries describe.
Assessment move
Create a small item bank for one construct, then annotate each item with the exact behaviour or judgement it represents. Ask a peer to explain every item aloud and describe how they chose an option. Revise wording when their interpretation differs from yours. Keep administration and sampling notes beside the questionnaire rather than treating them as a later reporting task.
Begin with a construct map that separates dimensions likely to require different items. For every draft, underline the timeframe, identify whether the response asks about agreement, frequency or evaluation, and check that the option labels match that task. Split any statement that could reasonably receive two different answers. Remove praise, blame and assumed knowledge from the stem.
Conduct a cognitive pilot by asking participants to paraphrase the question, describe the information they recalled and explain how they selected an option. Record mismatches rather than merely asking whether the item was clear. Test the visual order and availability of not-applicable responses because administration design can force an inaccurate substantive answer.
Prepare a fieldwork ledger with invitations, reminders, starts, completions, exclusions and item-level missingness. Compare the achieved respondents with the sampling frame on variables relevant to the research question. Decide before collection how partial questionnaires and duplicate entries will be handled. Finally, write the planned meaning of every scale score and the evidence needed before combining items.
A numerical total should not exist only because several statements share the same response options.
Working through Survey Instruments and Administration in EDS6003? Sia is AskSia’s AI Education tutor — ask any EDS6003 Survey Instruments and Administration question and get a clear, step-by-step explanation grounded in how EDS6003 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
