An Examination of SMS-Related Nonresponse Bias

Authors

Matthew Hastings, University of Nebraska-LincolnFollow

First Advisor

Kristen M. Olson

Date of this Version

10-2017

Document Type

Article

Citation

Hastings, M. J. (2017). An examination of SMS-related nonresponse bias (Doctoral dissertation).

Comments

A DISSERTATION Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfillment of Requirements For the Degree of Doctor of Philosophy, Major: Survey Research & Methodology, Under the Supervision of Professor Kristen M. Olson. Lincoln, Nebraska: October, 2017

Copyright (c) 2017 Matthew J. Hastings

Abstract

With the proliferation of mobile information and communications technologies, researchers face new opportunities for data collection and challenges to data quality. Short message service (SMS) or “text messaging” is a flexible mobile data service that can be incorporated into survey designs in a variety of ways. Given the many uses of SMS, I provide a framework for the use of SMS in the survey process which outlines the temporal location of three types of SMS-related nonresponse: SMS nonconsent, SMS nondelivery, and SMS noncooperation.

To better understand when SMS-related nonresponse might pose a risk of producing bias in survey estimates, I create three conceptual models of the mechanisms for SMS-related nonresponse – one for each of the three types of SMS-related nonresponse. Two forms of SMS-related nonresponse bias are analyzed in this dissertation, namely SMS nonconsent and SMS noncooperation. I examine the relative impact of these two forms of SMS-related nonresponse bias on a series of national estimates. Additionally, I create nonresponse weighting adjustments and examine their effectiveness at reducing SMS-related nonresponse bias in survey estimates.

This dissertation uses data collected from a SMS experiment conducted by the Gallup Organization from a pool of respondents to Gallup Daily surveys from July 29, 2013 – October 14, 2013. This design provides a rich sampling frame from which to examine variables available for both respondents and nonrespondents to the SMS surveys. I develop two sets of response propensity models – one set predicting SMS consent and the other predicting SMS cooperation. Using the predicted probabilities from these models, I examine the relationships between response propensity and a group of survey variables of interest.

Results indicate the presence of SMS-related nonresponse bias for a series of national survey estimates. However, the magnitude of bias differs across nonresponse types and across the survey variables of interest. Total SMS-related nonresponse bias is largely driven by noncooperation with the SMS survey. Results of the weighting adjustments were mixed. They performed well at reducing SMS nonconsent bias, but were less effective for SMS noncooperation. For both SMS nonconsent and SMS noncooperation, the strongest mechanisms of nonresponse tend to be respondent characteristics.

Adviser: Kristen M. Olson