Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Development of high -stakes credentialing examination test specifications using statistical models to compare with the test specifications from a Holistic Judgmental Model
Abstract
This research examined the comparability of three different statistical models and a Holistic Judgmental Model to develop test specifications for a high-stakes credentialing examination with four unique sections. Results from the credentialing body's recent practice analysis were used as the basis to develop test specifications for a new credentialing examination. The credentialing body's practice analysis survey was developed using ratings scales of importance and frequency. Using statistical models, data analyses were performed to evaluate various models of combining ratings scales of importance and frequency to generate test specifications. The statistical models used include the Additive Model, Multiplicative Model, and Hierarchical Ranking Scheme (Raymond & Neustel, 2006). Holistic Judgmental Model test specifications were developed with input from subject-matter experts. The results of this study indicate that test specifications developed using the Holistic Judgmental Model yield test specifications different from those generated using statistical models for three of four examination sections. For the fourth examination section, minor differences were noted between test specifications generated using statistical models and those developed using the Holistic Judgmental Model. Compelling similarities were found among test specifications generated by the three statistical models. In addition, no individual statistical method generated test specifications that were more comparable than another statistical model to the Holistic Judgmental Model. Based on results of the research, alternate test specifications could be selected for three of four examination sections if one or more statistical models had been chosen as the basis for development of test specifications. In addition, this research demonstrates that there should be a proportional distribution between the number of practice analysis survey statements in content areas, and the importance of these content areas identified by practitioners in the field in which the practice analysis is being conducted.
Subject Area
Educational psychology|Quantitative psychology
Recommended Citation
Rodeck, Elaine M, "Development of high -stakes credentialing examination test specifications using statistical models to compare with the test specifications from a Holistic Judgmental Model" (2008). ETD collection for University of Nebraska-Lincoln. AAI3331414.
https://digitalcommons.unl.edu/dissertations/AAI3331414