Title: A comparison of five methods for pretest item selection in online calibration

Authors: Yi Zheng; Hua-Hua Chang

Addresses: Division of Educational Leadership and Innovation, Mary Lou Fulton Teachers College, Arizona State University, 1050 S. Forest Mall, Tempe, AZ 85281, USA ' Department of Educational Psychology, University of Illinois at Urbana-Champaign, 1310 S. 6th St, Champaign, IL 61820, USA

Abstract: Many long-term testing programs rely on large item banks that need to be replenished regularly with new items, and these new items need to be pretested before being used operationally. Online calibration is a pretesting strategy in computerised adaptive testing, which embeds pretest items in operational tests and adaptively matches the pretest items with examinees. This paper compares five existing methods for pretest item selection in online calibration. A simulation study was conducted under the one-, two-, and three-parameter logistic models. The effects of two estimation methods, three seeding locations, and five calibration sample sizes were also investigated. Findings from the simulation study are mixed. However, overall, the simplest random selection method appears to be a potential best choice.

Keywords: computerised adaptive testing; item parameter estimation; online calibration.

DOI: 10.1504/IJQRE.2017.086500

International Journal of Quantitative Research in Education, 2017 Vol.4 No.1/2, pp.133 - 158

Received: 10 Oct 2016
Accepted: 23 Feb 2017
Published online: 10 Sep 2017 *