Sociology, Department of
Date of this Version
2-26-2019
Document Type
Article
Citation
Presented at “Interviewers and Their Effects from a Total Survey Error Perspective Workshop,” University of Nebraska-Lincoln, February 26-28, 2019.
Abstract
In an interviewer-administered survey, interviewers themselves can have a substantial impact on survey quality. In their review of the literature, West and Blom (2017) described the innumerable articles dedicated to describing this effect. A subset of these have provided support for the linkage between interviewer variation, such as differences in experience level and contacting approach, and variation in unit noncontact and nonresponse rates; these include Purdon, Campanelli, and Sturgis (1999), Groves and Couper (1998), and Blom (2012). In telephone studies, automated call scheduling systems help to mitigate these effects by using algorithms to ensure that cases receive an appropriate number of calls at optimal time points. In field studies, by contrast, field interviewers are often free to schedule calls and in-person visits based on their schedule and experience, creating opportunities for greater inconsistency of methodology applied across cases and, subsequently, Total Survey Error.
This presentation will describe our experiences implementing a case ownership approach to telephone contacting on a national study of young adults. In a case ownership model, telephone interviewers are each assigned a set of individual cases and encouraged to use their judgment to determine how best to work the case, similar to how field interviewers work their caseloads. This approach was implemented due to cost and schedule constraints on the project, which prevented the use of a Computer Assisted Telephone Interviewing (CATI) system to schedule and manage cases.
The study was conducted in late 2016 and early 2017, and approximately 2,000 sample members were included in the telephone contacting effort. Interviewers were responsible for case management, including determining which of their assigned cases to call and when, which roster lines to attempt, how often to schedule callbacks; gaining cooperation from sample members or sample members’ parents or guardians; and administering a web-based consent form over the phone, which took approximately 5 minutes to complete, or prompting the respondent to complete the self-administered web form. The interviewers were provided with talking points and suggested call scripts for various scenarios, but were not required to follow a verbatim script when gaining cooperation. The cases were initially distributed among interviewers using random assignment, although adjustments to case assignments were made throughout data collection. Limitations of the study include a lack of an experimental or fully interpenetrated design for interviewer case assignment.
This presentation will describe our experiences implementing the case ownership approach, including our methods for training interviewing staff on new case management skills. We will also describe the challenges we encountered, such as difficulties conducting case review and monitoring interviewer performance. We will analyze observed differences in interviewer contacting behavior, describe the magnitude of variability among interviewers related to the number and timing of calls and success gaining cooperation, and discuss the implications of this variation on survey error related to unit nonresponse. The primary purpose of this presentation is to describe the magnitude of the observed effects, but we will also attempt to correlate differences with interviewer characteristics to the extent possible.
Comments
Copyright 2019 by the authors.