Farklı Seçkisiz Madde Örnekleme Yaklaşımlarına Göre Angoff Ve Evet/Hayır Yöntemleri İle Belirlenen Geçme Puanlarının Karşılaştırılması

Abstract

In this study, the efficiency of various random sampling methods in order to reduce the number of items rated by judges in a standard setting study were examined and those methods were compared with each other. Those comparisons were made with respect to Angoff and Yes-No methods separately. Firstly, full-length test was formed by combining SBS 2012 and 2013 mathematics subsets. After then, simple random sampling (SRS), content stratified (C-SRS), item-difficulty stratified (D-SRS) and content-by-difficulty random sampling (CD-SRS) methods were used in order to constitute different length of subsets (30%, 40%, 50%, 70%) from the full-test. In total, 16 different study conditions (4 methods x 4 subsets) were investigated. In data analysis part, ANOVA analysis were conducted in order to examine whether minimum passing scores (MPSs) for the subsets were significantly different from the MPSs of the full-length test. As a follow up analysis, RMSE and SEE (Standard Error of Estimation) values were calculated for each study condition. Results indicated that only for 30% C-SRS, 40% C-SRS, 30% D-SRS and 30% CD-SRS study conditions, estimated Angoff MPSs were significantly different from the full-test Angoff MPS (45.12). In addition to the conditions above, 50% CD-SRS were also had an estimated Yes-No MPS significantly different from the full-test Yes-No (56.25). According to RMSE values, C-SRS method had smallest error while SRS method had biggest one for both Angoff and Yes-No cases. Moreover, SEE examinations revealed that in order to achieve estimations similar to the full-test Angoff MPS (within one SEE), it is sufficient to get 50% of items with C-SRS method. On the other side, similar estimations to the full-test Yes-No MPS could not been obtained in any of 16 study conditions. In conclusion, C-SRS method were more effective one compared to the others in reducing the number of items rated by judges in MPS setting studies conducted with Angoff and Yes-No methods.