Sociology, Department of
Date of this Version
2-26-2019
Document Type
Article
Citation
Presented at “Interviewers and Their Effects from a Total Survey Error Perspective Workshop,” University of Nebraska-Lincoln, February 26-28, 2019.
Abstract
Interviewer experience has been identified as an important factor in achieving higher response rates for telephone interviews. The causal mechanisms underlying this relationship remain unclear (Couper and Groves 1992; Jackle, Lynn, Sinibaldi and Tipping 2013; West and Blom 2016), but extant research suggests a combination of experience and personality traits, skills and attitudes explain substantial variation in cooperation rates (Groves and Couper; Jackle et al 2013). West and Blom (2016) summarize the positive relationship between experience and response rates and call for work to identify additional mediators of these relationships. The relationship between experience and data quality is even less clear, with research results split between evidence of both positive and negative relationships (West and Blom 2016).
While higher experience levels can lead to higher response rates, this investigation assesses the relationship between response rates and data quality; do more experienced interviewers gather higher numbers of unit responses but sacrifice higher data quality? This study examines interviewer experience in a computer-assisted telephone interview (CATI) setting. Specifically, we will examine the number of hours interviewers are actively engaged in dialing and speaking to respondents, interviewers’ response rates (the number of completed interviews divided by the hours worked) as well as data quality (item nonresponse in the form of number of refused responses or recorded responses of “Don’t Know” (DK) per completed interview). Initial exploratory analyses from 36 interviewers working on one statewide dual-frame telephone survey revealed a statistically significant correlation between the number of hours worked and our measurement of data quality. Interestingly, the direction of this correlation showed that as interviewer hours increase, so did the ratio of DK responses and refusals per completed survey (Pearson correlation = 0.38, p < 0.05).
Analysis of effects on data quality using a general linear model showed significant differences in data quality between five binned categories of interviewer hours (ranging from low to high). The adjusted R2 for the model was 0.531, indicating a high level of explanation for the variance seen in data quality. Similar analyses will be completed for 3 other dual-frame statewide telephone surveys to allow for replication of the findings across multiple interviewers and topics. Additional hierarchical multiple regression analyses will extend the scope of the study to include additional variables such as respondent demographic characteristics that could also influence data quality (the propensity for DKs and refusals) or interact with interviewer experience to impact data quality.
Comments
Copyright 2019 by the authors.