Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Item Response Theory-based approaches for computing cut scores from an Angoff -based standard setting study
Abstract
Even when the scoring of the examination is based on Item Response Theory, standard setting methods seldom use this information directly when determining the cut score for the examination from an Angoff-based standard setting study. Typically, the item performance estimates provided by the judges are simply added, regardless of the item parameter estimates for these items. Once these individual judges' totals are determined they are averaged across judges to determine the recommend cut score for the examination. When IRT scoring is used, this total score on the test can be converted to an IRT-based theta ability estimate by using the test characteristic curve. The purpose of this study was to investigate two approaches and its variations to determining the cut score from an Angoff-based standard setting study when the test questions have been calibrated using IRT methods. The first approach (called the Judge Theta Approach) converted the judges' item performance estimates (i.e., probability of correct response) to a theta estimate based on a 3PL IRT model. These theta estimates were then combined in various ways to calculate the cut score. The second approach (called the Simulated Item Vector Approach) used 0/1 values for each item that was determined from the judges' item performance estimates. Again, these 0/1 values were then used a variety of ways to generate a 3PL IRT-based cut score. Both variations of Judge Theta and variations 2 and 3 of Simulated Item Vector approach produced promising results.
Subject Area
Educational psychology
Recommended Citation
Ferdous, Abdullah A, "Item Response Theory-based approaches for computing cut scores from an Angoff -based standard setting study" (2005). ETD collection for University of Nebraska-Lincoln. AAI3167457.
https://digitalcommons.unl.edu/dissertations/AAI3167457