Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings

Authors

Jolene D. Smyth, University of Nebraska-LincolnFollow
Glenn D. Israel, University of FloridaFollow
Milton G. Newberry III, University of Georgia
Richard G. Hull, Nebraska Wesleyan UniversityFollow

Date of this Version

2019

Citation

Field Methods 31:3 (2019), pp. 260–276.

doi: 10.1177/1525822X19860648

Comments

Copyright © 2019 Jolene D. Smyth, Glenn D. Israel, Milton G. Newberry III, and Richard G. Hull. Published by SAGE. Used by permission.

Abstract

Considerable research has examined the effect of response option order in ordinal bipolar questions such as satisfaction questions. However, no research we know of has examined the effect of the order of presentation of concepts in the question stem or whether stem order moderates response option order. In this article, we experimentally test the main and interaction effects of both stem and response option order for items in self-administered surveys on response distributions and answer changes in eight satisfied/dissatisfied questions. We find consistent evidence that response option order impacts answers. We also find that the order of “satisfied” or “dissatisfied” in the question stem impacts response distributions for four of our eight items but does not moderate the effect of response option order. We discuss the implications of our findings for questionnaire design and secondary data analyses.