USING STATISTICS: Spotting and Avoiding Them
Introduction Types of Mistakes Suggestions Resources Table
of Contents About
Entire books have been written on the subject of wording of
Two main types of problems occur:
- Questions may be ambiguous. The responder may interpret them
differently than the questioner, and different responders may have
- The wording or sequencing of questions may influence the
- It is important to pilot test and modify questions before
- When reporting results of a survey, be sure to provide
access to the exact questions asked, so others can verify whether or
the questions are likely to be ambiguous or influential.
- Be cautious in interpreting the results of a survey. In
particular, try to find the exact questions asked and check them over
for possible ambiguity or other problems in wording. If the authors of
the survey are not willing to reveal the questions, be doubly cautious
in making interpretations. (Of course, there are other reasons to be
cautious of results of surveys in addition to possible problems with
question wording. For example, sampling, response rate, and method of
analysis of results are important.)
1. See, for example:
Converse JM and Presser S. (1986).
Survey Questions: Handcrafting the
Standardized Questionnaire, Sage
Fowler FJ Jr and Fowler FJ (1995). Improving
Survey Questions: Design and Evaluation, Sage.
Schroeder YC (1987). The
ethical ramifications of pretesting survey questions, Amer. J. of Trial Advocacy, vol 11,
pp. 195 - 201.
For additional sources of error in surveys, see
Beimer P and L Lyberg (2003), Introduction to Survey Quality,
updated July 31, 2012