Sociology, Department of
Designing Studies for Comparing Interviewer Variance Components in Two Groups of Survey Interviewers
Date of this Version
2-26-2019
Document Type
Article
Citation
Presented at “Interviewers and Their Effects from a Total Survey Error Perspective Workshop,” University of Nebraska-Lincoln, February 26-28, 2019.
Abstract
Methodological studies of interviewer effects often seek to identify factors that influence the magnitude of interviewer variance for particular survey questions. There is a long history of work in this area, and results from studies like this have informed current interviewing practice. Unfortunately, many studies of this type suffer from one or more of the following limitations in terms of their designs: 1) a failure to randomly assign interviewers to the treatments being compared; 2) a failure to formally test for differences in the variance components between the two groups; and 3) insufficient statistical power for comparison of the variance components. This paper (and the presentation at the interviewer effects workshop) will outline “ideal” designs for these types of studies, and present an example of successful implementation of the “ideal” design in practice. Statistical models enabling formal testing for differences in variance components will be introduced and illustrated using easy-to-use multilevel modeling procedures, and a SAS macro for performing power analyses when designing these studies will also be outlined.
Comments
Copyright 2019 by the authors.