Sociology, Department of
Questions Administered by Telephone or In Person: Differences in Interviewer-Respondent Interactions
Date of this Version
2-26-2019
Document Type
Article
Citation
Presented at “Interviewers and Their Effects from a Total Survey Error Perspective Workshop,” University of Nebraska-Lincoln, February 26-28, 2019.
Abstract
When choosing a mode for data collection of computer-assisted surveys, a researcher has three main options available: the computer-assisted telephone interview (CATI), the computer-assisted personal interview (CAPI) or a web interview (i.e., a self-adminstered interview). Generally, CAPI allows for collecting most complex data, of the highest quality, but only when interviewers are well-trained and effort is made to monitor and manage interviewers during field work. This higher data quality in CAPI interviews may be due to the finding that presence of an interviewer reduces the amount of respondents’ satisficing behaviors (i.e., not investing the required effort to provide meaningful answers, see Heerwegh 2008). An interviewer can motivate respondents, presumably by means of rapport (Garbarski et al 2016) to invest the amount of cognitive effort required to answer questions thoughtfully. However, with interviewers administering the survey their social presence may also give rise to respondents’ reflection of social consequences of providing information to a relative stranger. By looking at response distributions it has been shown that social desirability bias and satisficing are more prevalent in CATI than in CAPI (see Holbrook et al 2003), and lowest in (self-administered) Web interviews. To fully explain such effects, it makes sense to study actual behaviors in interviewer-respondent interactions. In this approach, generally referred to as interaction coding or behavior coding (see Ongena and Dijkstra, 2006), interactions are systematically evaluated on deviations of the so-called paradigmatic sequence. A paradigmatic sequence is the interaction as intended by the researcher, with a sequence consisting of only two or three actions: the interviewer reading the question exactly as worded, the respondent providing an answer that exactly matches one of the response options, and optionally, an interviewer acknowledgement. Any deviation from this sequence may indicate problems in the questionnaire or the interviewing procedure. However, to our knowledge, CATI and CAPI interactions have never been systematically compared. By analyzing interactions of 60 CATI and 54 CAPI interviews that originated from a mixed-mode experiment using the European Social Survey questionnaire (Haan 2015), we found mixed differences with respect to behaviors in CATI and CAPI interactions. For example, interviewer laughter appeared to be more common in CATI than in CAPI, but apologetic utterances such as ‘sorry’ occurred equally often in both modes. Furthermore, a significant difference was found in the number of words uttered. Question-answer sequences contained more words in CATI than in CAPI. This is partly explained by the fact that for many questions in the CAPI survey show cards were used. Lack of show cards in CATI extends interactions due to less efficient communication about response alternatives. Further analysis showed that respondents in CATI had more difficulty in formulating their response than in CAPI. These task-related issues may contribute to the effect of decreased trust and motivation of respondents in CATI interviews, and may subsequently explain the increased level of satisficing and social desirability bias in this survey mode compared to CAPI.
Comments
Copyright 2019 by the authors.