Sociology, Department of
Document Type
Article
Date of this Version
Spring 2006
Citation
Public Opinion Quarterly 70:1 (Spring 2006), pp. 66–77.
doi: 10.1093/poq/nfj007
Abstract
For survey researchers, it is common practice to use the check-all question format in Web and mail surveys but to convert to the forced-choice question format in telephone surveys. The assumption underlying this practice is that respondents will answer the two formats similarly. In this research note we report results from 16 experimental comparisons in two Web surveys and a paper survey conducted in 2002 and 2003 that test whether the check-all and forced-choice formats produce similar results. In all 16 comparisons, we find that the two question formats do not perform similarly; respondents endorse more options and take longer to answer in the forced-choice format than in the check-all format. These findings suggest that the forced-choice question format encourages deeper processing of response options and, as such, is preferable to the check-all format, which may encourage a weak satisficing response strategy. Additional analyses show that neither acquiescence bias nor item nonresponse seem to pose substantial problems for use of the forced-choice question format in Web surveys.
Comments
Copyright © 2006 Jolene D. Smyth, Don A. Dillman, Leah Melani Christian, and Michael J. Stern. Published by Oxford University Press on behalf of the American Association for Public Opinion Research. Used by permission.