Developing a Quality Control Protocol for Evaluation of Recorded Interviews

Authors

Margaret Hudson, Survey Research Center, University of MichiganFollow
Lisa Holland, Survey Research Center, University of MichiganFollow
Lisa Lewandowski-Romps, Survey Research Center, University of MichiganFollow

Date of this Version

2-26-2019

Document Type

Article

Citation

Presented at “Interviewers and Their Effects from a Total Survey Error Perspective Workshop,” University of Nebraska-Lincoln, February 26-28, 2019.

Comments

Copyright 2019 by the authors.

Abstract

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, with a discussion of how the data were used to inform further development of the evaluation protocol. Although some aggregate data will be shared, the presentation will largely focus on the operational considerations related to the development and implementation of the quality control protocol across projects.