Sabit ve Anında Bireyselleştirilmiş Çok Aşamalı Testlerin Karşılaştırılması

Abstract

Many large-scale assessments have used adaptive testing approaches such as Computerized Adaptive Testing (CAT) and Computerized Multistage Testing (MST). The MST approach has been preferred instead of the CAT with the idea that it can reduce the problems of the CAT such as under or over estimating ability, not allowing movement between items, requiring an answer to skip an item, the impossibility of using common-root items consisting of scenario-based questions, and the perception of inequality due to participants taking different tests. Although the MST reduces the problems of the CAT with panels and modules that are combined before implementation, it has the disadvantages of having fewer adaptation points, and the test and item security threats caused by the continuous application of the same panel and module structures. The On-the-fly MST, a new approach in which items are combined at the participant's ability level to form modules, has potential as it utilizes the advantages of the CAT and MST and reduces their weaknesses. The aim of this study is to compare F-MST and O-MST approaches in terms of measurement accuracy and item security under different simulation conditions. A simulation study was conducted in the research. An item pool was created by generating the parameters of 400 items with the parameter distributions of the items based on the 3PL model used in the TIMSS application. In the study, comparisons were made with 100 replications under 72 simulation conditions. The results show that the O-MST produces better results than the F-MST in terms of both measurement precision and item security. For short test lengths, the A-BCAT has considerably better measurement accuracy than the F-MST. According to ability distributions, it was concluded that the O-MST produced better results than the F-MST, especially in non-normal distributions. According to the module/test length ratio, it was observed that the difference in the measurement accuracy of O-MST increased in the K-K-U ratio where the last module length increased. The successful measurement accuracy and item security findings of the O-MST are discussed in the light of both large-scale evaluations and the literature.