A Comparison of Fully Labeled and Top-Labeled Grid Question Formats

Authors

Jolene Smyth, University of Nebraska-LincolnFollow
Kristen M. Olson, University of Nebraska - LincolnFollow

Date of this Version

2020

Citation

Smyth, Jolene and Kristen Olson. 2019. “A Comparison of Fully Labeled and Top Labeled Grid Question Formats.” Edited by P. Beatty, D. Collins, L. Kaye, J. Padilla, G. Willis, and A. Wilmot. Chapter 10 in Advances in Questionnaire Design, Development, Evaluation and Testing. Wiley. Pp. 229-258.

Comments

Copyright © 2020 John Wiley & Sons, Inc. Used by permission.

Abstract

The grid question format is common in mail and web surveys. In this format, a single question stem introduces a set of items, which are listed in rows of a table underneath the question stem. The table’s columns contain the response options, usually only listed at the top, with answer spaces arrayed below and aligned with the items (Dillman et al. 2014).This format is efficient for respondents; they do not have to read the full question stem and full set of response options for every item in the grid. Likewise, it is space efficient for the survey researcher, which reduces printing and shipping costs in mail surveys and scrolling in web surveys.

However, grids also complicate the response task by introducing fairly complex groupings of information. To answer grid items, respondents have to connect disparate pieces of information in space by locating the position on the page or screen where the proper row (the item prompt) intersects with the proper column (the response option). The difficulty of this task increases when the respondent has to traverse the largest distances to connect items to response option labels (down and right in the grid) (Couper 2008; Kaczmirek 2011).This spatial connection task has to be conducted while remembering the shared question stem, perhaps after reading and answering multiple items. As a result, grid items are prone to high rates of item nonresponse, straightlining, and breakoffs (Couper et al. 2013; Tourangeau et al. 2004).

One way to possibly ease the burdens of grids in mail surveys is to repeat the response option labels in each row next to their corresponding answer spaces (Dillman 1978). Including response option labels near the answer spaces eliminates the need for vertical processing, allowing respondents to focus only on processing horizontally. However, fully labeling the answer spaces yields a more busy, dense display overall, which one can speculate might intimidate or overwhelm some respondents, leading them to skip the grid entirely.

In this chapter we report the results of a series of experimental comparisons of fully labeled versus top-labeled grid formats from national probability mail survey, a convenience sample of students in a paper-and-pencil survey, and a convenience sample in a web-based eye-tracking laboratory study. For each experiment we compare mean responses, inter-item correlations, item nonresponse rates, and straightlining. In addition, for the eye-tracking experiment we also examine whether the different grid designs impacted how respondents visually processed the grid items. For two of the experiments, we conduct subgroup analyses to assess whether the effects of the grids differed for high and low cognitive ability respondents. Our experiments are conducted using both attitude and behavior questions covering a wide variety of question topics and using a variety of types of response scales.