Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
An investigation to determine whether an algorithm based on response latencies and number of words can be used in a prescribed manner to reduce measurement error
Abstract
Responding to all items on tests when there is no penalty for guessing is an often used testwiseness strategy. Even when correction for guessing formulae are employed, it is still advantageous for an examinee who can eliminate at least one distractor to guess. With this in mind, it certainly seems useful to devise a system to determine whether such guessing does, in fact, contribute to lower measurement precision. Computerized test administration facilitates the collection of latency data while examinees take a test. Two random samples, each containing 3,000 examinees, were drawn from the population of examinees who took Form J of the computer-administered GRE General Test administered between October 1, 1992 and September 30, 1993. An algorithm was developed to identify examinees who spent much less time than expected on certain items. In this algorithm, number of words and latency were used to compute an examinee's mean working rate on items from a particular content area of the test. This rate was then used in conjunction with the word count of later items of similar format from the same content area to compute an expected latency on these items. If the examinee spent less than a fraction of this latency on a particular item, the response was flagged and rescored as a not reached item. The algorithm was developed using Sample 1 data, and evaluated using Sample 2 data. Item Response Theory (IRT) calibrations, using PC-BILOG 3, were completed on both the original data and the rescored data, and the standard errors associated with the item and person parameter estimates before and after using the rescoring algorithm were compared and contrasted. Analyses of the original and rescored data revealed that, in general, across the three content areas, rescoring using the proposed algorithm did not result in improvements in the precision with which the IRT a-, b-, c-, and person parameter estimates were measured.
Subject Area
Educational evaluation
Recommended Citation
Bhola, Dennison Suresh, "An investigation to determine whether an algorithm based on response latencies and number of words can be used in a prescribed manner to reduce measurement error" (1994). ETD collection for University of Nebraska-Lincoln. AAI9519527.
https://digitalcommons.unl.edu/dissertations/AAI9519527