12. Computerized Testing In Licensure

Authors

C. David Vale, Insurance Testing Corporation

Date of this Version

1995

Document Type

Article

Citation

Licensure Testing: Purposes, Procedures, and Practices, ed. James C. Impara (Lincoln, NE: Buros Institute of Mental Measurements, University of Nebraska-Lincoln, 1995).

Comments

Copyright © 1995 by Buros Institute of Mental Measurements. Digital Edition copyright © 2012 Buros Center for Testing.

Abstract

Computerized testing has come out of the laboratory and into the field. By rough estimates, over a million licensure and certification examinations are currently given by computer each year, and the number is rising. Computerized testing is not appropriate for every application, however. Computerized tests always result in significantly greater direct costs than paper-and-pencil tests. To justify their use, a computerized test must result in a net dollar saving. This means that something in the process of computerization must offer a cost reduction that more than offsets the direct cost of computerization. The purpose of this chapter is to identify the areas in which computerization can result in dollar savings and to help the reader determine if, and in what form, computerized testing is appropriate for a specific application.

It may be possible to make the case that a computerized test is useful because it can implement new question types or questioning strategies and thus measure something that cannot be measured by other means. Such an application has yet to be demonstrated in licensing. This chapter will thus ignore this possibility, dealing exclusively with the use of computerization of traditional test questions as a means of saving costs.

SCHEDULING EFFICIENCY-AN OBVIOUS ADVANTAGE

The success of computerized testing in licensure today is due in large part to the scheduling improvements it has offered. Consider a typical paper-and-pencil license testing program: Tests are given every 2 weeks and must be scheduled 2 weeks in advance. Say a candidate decides on October 1 to take a licensure test. The scheduling deadline for the October 14 test has just passed and the first test available is October 28. The candidate takes and fails that test, learns of the failure on November 10, and must reschedule for November 25. A typical computerized testing program is different: Tests are given daily and candidates need to register only one day in advance. Thus, the candidate could fail the first test on October 2, study hard that night, and take the retest on October 3. Assuming the candidate passed the second time in either scenario, the result of computerization would be a time saving of almost 2 months. If passing a test stands between a candidate and a career, a 2-month time saving can be significant.

Why does a computerized testing program offer such scheduling improvements? The direct costs in a testing program can be divided into five categories: (1) registering a candidate to take a test, (2) providing a place for the candidate to take the test, (3) providing a medium on which to present the test, (4) providing someone to proctor the examination, and (5) scoring and reporting the results. An optimal administration design must balance all five of these categories. If the criterion for design is minimal cost, the least expensive combination of elements must be found.

Paper-and-pencil administration offers significant freedom to choose a low cost design. The minimal expense in administration is achieved by requiring the candidate to mail an application and a check (avoiding telephone and credit-card charges), administering the test in idle space that is normally used for other purposes (e.g., Saturday in a high-school cafeteria), presenting the questions on an inexpensive medium (e.g., paper), using part-time personnel earning supplemental (lower wage) income to administer the test, and limiting expensive equipment to a single site (e.g., scoring and reporting results from a central office). The optimal economic design results in the often seen massed administration of paper and pencil tests and 2- to 4-week advance registration requirements.

A computerized testing program has less freedom in design. The media for test presentation are not readily portable; this suggests implementation in a permanent site. The media, as well as the space to store them, are relatively expensive; this suggests that relatively few be used. When the costs of equipment and space are balanced against the cost of proctoring, small, frequent sessions usually result. In its optimal configuration, computerized administration is significantly more expensive than paper-and-pencil administration. Historically, this naturally gave rise to the offering to candidates of improved services such as rapid scheduling and score reporting.

Computerized administration is not essential to achieve the scheduling advantages typically obtained through computerized testing. However, when the design appropriate for computerization (and yielding the scheduling advantages) is applied to paper-and-pencil testing (e.g., small, frequent sessions; rapid scheduling; onsite score reporting), its costs are nearly as great as full computerization. The direct cost of a computer system adequate for implementing multiple-choice licensure tests is only about $300 per testing station per year, which translates to about one dollar per test in a center that gives one test per station per day. Thus, if daily testing is implemented, the additional costs of computerization are small.

Scheduling improvements, from a scientific perspective, are not very interesting. Psychometric journals rarely publish articles documenting the time saved through efficient handling of candidates. As a point of comparison with psychometric savings discussed below, however, remember that the time savings achieved through scheduling improvements are on the order of 1 to 2 months.

Note, however, that these time savings translate into dollar savings only when the time has value. Time typically has great value when a candidate must pass a test to get a license to practice a profession. When the translation is achieved by comparing the earning power of an unemployed individual with that of a licensed individual, the figures are large enough to defy belief. Anecdotal experience suggests that these savings are meaningful to licensure candidates. Time has less value if the candidate can practice the profession on a provisional license while attempting to pass the test. Similarly, time has less value to certification candidates than to license candidates because the connection between having the certification and earning money is less direct. If the decision to computerize a test is based on the improvements possible in scheduling efficiency, it is wise to first verify that the time saved is truly valuable.