Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
An Investigation of Item Calibration Approaches in Multistage Testing
Abstract
Multistage testing (MST) has gained popularity over the past decade for its combined advantages of linear tests and computerized adaptive tests. Many large-scale testing programs have implemented MST as test delivery mode. Studies have investigated the impact of various MST operational characteristics on measurement precision of ability estimation and classification accuracy. However, little research has examined the issue of item calibration in MST. The accuracy of item parameters is fundamental to MST testing programs because many aspects of test development rely on item parameters, including information functions, module selections, the estimation of final proficiency scores, etc. Item calibration in MST is a necessary procedure that ensures the accuracy of item parameters. This study investigates the performance of four item calibration approaches, separate calibration with linking (SCL), fixed common item parameter (FCIP) calibration, concurrent calibration assuming a single group of ability distribution (CCSG), and concurrent calibration with the multiple-group procedure (CCMG) in a 1–3 MST design. This research consists of a simulation study and a real data application. The simulated factors include routing module length, routing rule, and sample size. The feasibility and efficiency of the item calibration approaches were evaluated in terms of item and person parameter recovery, Pearson correlation coefficient between estimated and true abilities, and classification accuracy rate. The real data came from an eighth-grade mathematics assessment that had recently transitioned from a linear fixed form to an MST delivery mode. Psychometric outcomes from the pre-equating model and post-equating models based on each item calibration approach were compared. Results suggested that SCL might not be feasible, while FCIP and CCSG performed similarly well in MST item calibration. Meanwhile, CCMG was found to perform poorly in the conditions and the testing situation examined. This study informs the advantages and disadvantages of four item calibration approaches in MST and provides practical suggestions for testing programs that implemented MST. Limitations and directions for future studies are also discussed.
Subject Area
Education|Quantitative psychology
Recommended Citation
Cai, Liuhan Sophie, "An Investigation of Item Calibration Approaches in Multistage Testing" (2018). ETD collection for University of Nebraska-Lincoln. AAI10933991.
https://digitalcommons.unl.edu/dissertations/AAI10933991