Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
The exploration of an alternative method for scoring computer-adaptive tests
Abstract
The prospect of making computer adaptive testing (CAT) understandable to test-takers is challenging because CAT is grounded in IRT, which is based on scoring algorithms that are more complex than the scoring process in conventional testing (e.g., number-correct scoring). Moreover, explaining to test-takers why two individuals taking a CAT can answer the same item correctly, and yet each be given a different amount of "credit" for this correct response is also difficult. The purpose of this study was to explore the number-right method (CATNR) for scoring CATs, one which is less directly reliant on IRT. This alternative scoring method is based on the more familiar equated number-correct score and is identical to the procedures used currently to score and equate many traditional paper-and-pencil tests. If the CATNR scoring algorithm produces scores comparable to those produced by 3PL-IRT scoring, it may hold promise for rendering the explanation of adaptive testing to test-takers and the public more understandable. In Study 1, a comparability study for a linear version and CAT version of a high stakes standardized test was replicated using the CATNR method to rescore the CAT sections. Results indicated that the CATNR scoring approach closely replicated the results of the original comparability study in which CATS are scored using maximum likelihood estimation (MLE). In Study 2, CAT scores from an operational administration of the same test were also rescored CATNR and compared to the original CAT scores which were also based on MLE scoring. Again, results supported the psychometric adequacy of the CATNR scoring method. A very small number of large score differences between the CATNR and CAT scoring methods were obtained. However, further investigation revealed that the majority of these differences exist when the 3PL model does not fit the data. Limitations of this study and directions for further research are discussed, as well as issues that policy-makers must consider before the CATNR method is implemented.
Subject Area
Educational evaluation|Psychological tests
Recommended Citation
Potenza, Maria T, "The exploration of an alternative method for scoring computer-adaptive tests" (1994). ETD collection for University of Nebraska-Lincoln. AAI9510976.
https://digitalcommons.unl.edu/dissertations/AAI9510976