BİLGİSAYARDA BİREYSELLEŞTİRİLMİŞ TEST UYGULAMALARINDA KAPSAM DENGELEMENİN ÖLÇME KESİNLİĞİNE ETKİSİ
Özet
In this study, the aim is to examine how applying content balancing in computerized adaptive testing (CAT) affects measurement accuracy estimation values based on different sample sizes, ability estimation methods, item selection methods, and stopping rules. For this purpose, a 750-item pool was created using ability parameters for two different simulated groups of 250 and 500 individuals. This pool, weighted equally across five different content areas scored in two categories, was generated using the three-parameter logistic (3PL) item response model. CAT applications were conducted under conditions where content balancing was and was not applied for both 250- and 500-person samples, using different ability estimation methods, item selection methods, and test stopping rules. The CAT applications for the 80 conditions created in the study were compared using the “catR” package in the R programming language, with 50 replications for each condition. The simulations calculated RMSE, bias, and correlation values for measurement accuracy in each condition. Results indicated that, in general, content balancing led to slight increases in average error and bias values. Despite these small increases, a high correlation was observed between the estimated and actual ability levels across different test conditions with and without content balancing. Additionally, among the ability estimation methods included in the study, Expected a Posteriori method proved to be more effective in terms of RMSE, bias, and correlation values. The research also found that content balancing increased the required number of items by about one item to reach a specific accuracy threshold, especially under strict stopping rules where the standard error falls below a set threshold value.
