Biological Sciences, School of
First Advisor
Brian A. Couch
Date of this Version
Summer 6-14-2023
Citation
Uminski, C. (2023). Evaluating Assessment Score Validity and Characterizing Undergraduate Biology Exam Content. [Doctoral dissertation, University of Nebraska–Lincoln].
Abstract
The landscape of undergraduate biology education has been shaped by decades of reform efforts calling for instruction to integrate core concepts and scientific skills as a means of helping students become proficient in the discipline. Assessments can be used to make inferences about how these reform efforts have translated into changes in department curriculum and course practices. Such changes can be measured using student scores on researcher-developed programmatic and concept assessments. Scores on these assessments are often assumed to be accurate representations of student biology content knowledge, but my work indicates that the validity of these interpretations may be threatened when students complete the assessments in low-stakes contexts that are more likely to elicit low test-taking effort. Score validity is also threatened in high-stakes out-of-class contexts in which students may be incentivized to leverage external resources to increase their score. My findings suggest that departments and instructors using programmatic and concept assessments to evaluate the progress of their curriculum and courses in meeting the goals of reform effort should carefully interpret scores in light of the conditions in which students completed the assessment. The impacts of reform efforts may also be detected in the types of skills and content that are assessed on course exams. I studied the skills and content of lower-division undergraduate biology exams in the context of a three-dimensional framework consisting of scientific practices, interdisciplinary crosscutting concepts, and disciplinary core ideas. I found that very few exam items were three-dimensional, primarily due to the low number of items assessing scientific practices. Although there were few three-dimensional items, those items were more likely to use a constructed-response format and assess higher-order cognitive skills compared to items not aligned with all three dimensions. To achieve the goals of reform efforts in undergraduate biology education, my research indicates instructors may need time, resources, and training for writing and grading three-dimensional assessments. Altogether, this dissertation sheds critical insight into the process and content of evaluating student learning, thereby refining our understanding of the impact of education reforms.
Advisor: Brian A. Couch
Comments
A DISSERTATION Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfillment of Requirements For the Degree of Doctor of Philosophy, Major: Biological Sciences, Under the Supervision of Professor Brian A. Couch. Lincoln, Nebraska: June, 2023
Copyright © 2023 Crystal Uminski