Sociology, Department of

 

Document Type

Article

Date of this Version

2020

Citation

Published in Journal of Survey Statistics and Methodology 8 (2020), pp 636–666

doi: 10.1093/jssam/smz031

Comments

Copyright © 2019 Kristen Olson, Jolene D. Smyth, & Antje Kirchner. Published by Oxford University Press on behalf of the American Association for Public Opinion Research. Used by permission.

Abstract

Asking questions fluently, exactly as worded, and at a reasonable pace is a fundamental part of a survey interviewer’s role. Doing so allows the question to be asked as intended by the researcher and may decrease the risk of measurement error and contribute to rapport. Despite the central importance placed on reading questions exactly as worded, interviewers commonly misread questions, and it is not always clear why. Thus, understanding the risk of measurement error requires understanding how different interviewers, respondents, and question features may trigger question reading problems. In this article, we evaluate the effects of question features on question asking behaviors, controlling for interviewer and respondent characteristics. We also examine how question asking behaviors are related to question-asking time. Using two nationally representative telephone surveys in the United States, we find that longer questions and questions with transition statements are less likely to be read exactly and fluently, that questions with higher reading levels and parentheticals are less likely to be read exactly across both surveys and that disfluent readings decrease as interviewers gain experience across the field period. Other question characteristics vary in their associations with the outcomes across the two surveys. We also find that inexact and disfluent question readings are longer, but read at a faster pace, than exact and fluent question reading. We conclude with implications for interviewer training and questionnaire design.

Olson JSSM 2020 THE EFFECT OF QUESTION supplementary_data.zip (370 kB)
WLT & NOP Questionnaires + submitted & revised drafts

Share

COinS