Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.

Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.

A comparison of the accuracy of rational estimates of employment test validities and statistical models

Pamela Cherie Waits, University of Nebraska - Lincoln

Abstract

The present study is based on two earlier studies of rational validity estimation. These studies compared the accuracy of rational estimates of validity coefficients to empirical studies. The first study employed experts in the field of personnel psychology (Schmidt, Hunter, Croll, & McKenzie, 1983). The second study employed less experienced psychologists (Hirsh, Schmidt, & Hunter, 1986). The experts were more accurate than novices. The first portion of the present study addressed the issue of expert accuracy. Fourteen participants from the earlier studies were interviewed over the telephone about the process they used to estimate validity coefficients. No significant differences were found between experts and novices. Experts were variable in the strategies they used to make their estimates and their strategies overlapped with those used by novices. It was hypothesized that differences in accuracy between experts and novices were caused by knowledge of validity coefficients, statistical estimation, or both. The second portion of the present study compared the accuracy of rational validity estimates to statistical modesl derived from validity generalization data. Six models were derived based on varying levels of test-job combinations. Three job levels were defined using the first three digits of the DOT code. Two levels of test similarity were defined. First, tests were defined by the construct measured, and then constructs were grouped into measures of general intelligence or specific knowledge. For each level of test-job combination, both the levels and patterns of validity coefficients were compared to the true validities. The level of expert estimates were slightly better than validity generalization. In addition, validity generalization was comparable to the accuracy of novice estimates. The pattern analysis revealed that validity generalization is better than rational estimates in predicting the pattern of true validities. It was concluded that rational estimates provide unque information in the estimation of levels of employment test validities, especially when the judges are selectively chosen. In addition, validity generalization estimates were extremely accurate in estimating relative validities. The results suggest that validity generalization is applicable in more situations than originally thought.

Subject Area

Occupational psychology

Recommended Citation

Waits, Pamela Cherie, "A comparison of the accuracy of rational estimates of employment test validities and statistical models" (1987). ETD collection for University of Nebraska-Lincoln. AAI8806156.
https://digitalcommons.unl.edu/dissertations/AAI8806156

Share

COinS