Date of this Version
McBrien, S.B. (2018). Effects of Structural Flaws on the Psychometric Properties of Multiple-Choice Questions.
The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties of the items. Structural flaws were identified by educational support staff instead of the faculty experts who authored the items and were responsible for teaching the content knowledge the items were intended to assess. Two-way ANOVA was used to measure the outcome of the revision project and structural flaw type on the psychometric qualities of existing conventional multiple-choice examination items. Neither variable had a statistically significant effect on the psychometric qualities of the items. Nonetheless, efforts to remove structural flaws from multiple-choice items may lead to stronger reliability estimates, enhanced validity evidence, and an improved test-taking experience for students.
Adviser: Allen Steckelberg