Effects Of Item Pool Characteristics On Ability Estimate And Item Pool Utilization: A Simulation Study

Abstract

Forming an item pool for computerized adaptive testing requires a long and demanding process that may be challenging, both in terms of time and cost. Therefore, one may come across such questions as `How should an optimal item pool be?' and/or `How many items should exist in an item pool?' Although research with regard to the features to exist in an optimal item pool vary, there has been no consensus reached about how big the item pool size should be. In the current study, the effect of different item pool size and item distribution on ability estimation and item pool utilization was analysed. 36 different item pools were generated through SimulCAT software. Using 1,000 simulees, single session CAT environments were simulated and two different termination rules were used in the study. Findings of the study indicated that as the size of the item pool increased to a specific size, the precision of measurement increased and the number of unused items decreased. By examining the results according to b parameter, it was found that the effect of b parameter distribution over the results decreased.