ENVM7003 Chap.5 Question Wording That Biases the Answer
Question Wording That Biases the Answer
Badly written items are not bad in general. They are bad in thirteen specific, recognisable ways, and the course supplies a poor and better pair for each.
Once the fault has a name the repair is nearly mechanical; without a name you rewrite by instinct and usually introduce a second fault.
The catalogue runs from vague phrasing and jargon through imprecision, bias from a supplied context, unequal comparison between options, unbalanced response sets, loaded tone, items too difficult to answer, double-barrelled items, overlapping options, assumed knowledge, an unworkable time frame, categories that will not line up with published statistics, and cryptic phrasing.
This chapter names each fault, works four repairs in the three-part form the weekly task wants, and then turns to the harder skill: reading the response pattern after fielding to work out which fault you missed. That diagnosis feeds a limitations paragraph that says specifically how much of the argument survives, which is worth far more than a general admission that the study was small.
What this chapter covers
- 01
5.1 Thirteen named faults and the repair move for each
- 02
5.2 Four repairs written as fault, effect on data, revision
- 03
5.3 Screening for awareness before asking for a judgement
- 04
5.4 Reading the response pattern to find the fault you missed
- 05
5.5 Two patterns that look like faults and are not
- 06
5.6 Writing a limitation that says what still stands
Repairing an unbalanced response set and a double-barrelled item
- +2Item one, as written. Five options running too high, about right, slightly too low, moderately too low, far too low.
- +3The fault and its effect. Unbalanced response choices: one point sits above the midpoint and three below, so the average drifts downward by construction and a movement in the mean cannot be interpreted.
- +2The revision. Far too high, slightly too high, about right, slightly too low, far too low. Equal distance either side of a genuine centre.
- +2Item two, as written. Do you support protecting mangroves but not restricting boat access? A respondent who supports both cannot answer at all.
- +1The revision. Two separate items with their own response sets, which also lets you report the interesting group: those who want one protection and not the other.
Key terms
- Double-barrelled item
- A question containing two propositions sharing one answer, so a respondent who agrees with one and not the other has no honest reply available.
- Unbalanced response set
- A closed set with more options on one side of the midpoint than the other, which shifts the average by construction rather than by respondent opinion.
- Context bias
- Distortion introduced by a preamble that supplies a norm or a reason before the question is asked. The repair is to delete the preamble and ask the question cold.
- Assumed knowledge
- A fault where the item presumes awareness of a policy or event. Respondents answer anyway, so the repair is a screening pair before the judgement item.
- Mutually exclusive options
- A response set where no two options can be true at once. Where overlap is real, either split the dimensions or state explicitly that multiple answers apply.
- Commensurate categories
- Bands that align with the published statistics you intend to compare against. They must be set before fielding, because they cannot be realigned afterwards.
Question Wording That Biases the Answer FAQ
What is a leading question and how do I avoid one?
A leading item signals the approved answer through its wording, its preamble or its options. Three of the named faults produce that effect: tone bias from loaded adjectives, context bias from a preamble supplying a norm, and unbalanced response sets. Read the item aloud and ask which answer a respondent would give to be agreeable; if there is one, the item is leading.
Can I fix a bad question after I have collected responses?
No, and that is why the catalogue matters before fielding. What you can do afterwards is diagnose it and say precisely what the data still supports. A limitation naming the specific mechanism and pointing to an unaffected item is worth marks; a general statement about limitations is not.
Everyone answered my item the same way. Is that bias or a real finding?
Rule out the instrument before concluding anything about the population. Check the stem for approving adjectives and a supplied norm, and count the options either side of the midpoint. A genuine consensus usually shows up across several items about the same thing, whereas a wording fault shows up on one item while its neighbours behave normally.
Why do people skip particular questions?
Most often because the item is too difficult or too intrusive. An item asking for the share of yearly water that goes outdoors demands a calculation almost nobody can perform, and an item asking for identifying detail invites people to stop. High skip rates on one item belong in your limitations and usually point at one of those two causes.
Assessment move
Take any real instrument you can find and run the catalogue over it, naming a fault per item where one exists. Twenty minutes of that builds the pattern recognition faster than reading the list again, and it is directly transferable to the peer feedback you owe another group.
When you draft your own items, do the balance check and the single-proposition check as two separate passes, because they catch different things and doing them together means doing neither properly.
Organisational Behaviour · Identifying and Assisting Students at Risk · Digital Marketing and Social Media
