Computer Science and Engineering, Department of


Date of this Version



Paper Session: Computational Thinking Assessment SIGCSE ’20, March 11–14, 2020, Portland, OR, USA


2020 Copyright is held by the owner/authors


Calls for standardized and validated measures of computational thinking have been made repeatedly in recent years. Still, few such tests have been created and even fewer have undergone rig- orous psychometric evaluation and been made available to re- searchers. The purpose of this study is to report our work in de- veloping and validating a test of computational thinking concepts and skills and to compare different scoring methods for the test. This computational thinking exam is intended to be used in com- puting education research as a common measure of computational thinking so that the research community will be able to make more meaningful comparisons across samples and studies. The Computational Thinking Concepts and Skills Test (CTCAST) was administered to students in several courses, evaluated and revised, and then administered to another group of students. Part of the revision included changing half of the items to a multiple-select format. The test scores using the three scoring methods were com- pared to each other and to scores on a different test of core com- puter science knowledge. Results indicate the CTCAST and the test of core computer science knowledge measure similar, but not identical, aspects of students’ knowledge and skills, and that item- level statistics vary according to the scoring method that is used. Recommendations for using and scoring the test are presented.