Çok Kategorili Puanlanan Maddelerde Değişen Madde ve Adım Fonksiyonu Belirleme Yöntemlerinin İncelenmesi

Abstract

The aim of this study is to compare the polytomous differential item functioning (DIF) and differential step functioning (DSF) detection methods in polytomously scored items. In this context, it was studied Turkey, Kazakhstan and the USA samples responding to items (IC011) related to frequency of using digital devices in "ICT Familiarity Questionnaire" which was administered to students in within the scope of PISA 2018. Mantel test, Liu-Agresti statistics, Cox β and poly-SIBTEST methods for multi-category DIF analysis; Adjacent Category Log Odds Ratio (AC-LOR) and Cumulative Category Log Odds Ratio (CU-LOR) methods were used for DSF analyzes. The research model is relational screening and conditions such as "category combination rule, focus group-reference group, focus group sample size, sample rating of focus group: reference group, DIF investigation method" were created. In the creation of the conditions, from SAS and R software; in analysis, SIBTEST for poly-SIBTEST method; DIFAS programs were used for other methods. As a result of the analysis, the number of item / steps with high levels of DIF / DSF; it was observed that it was more in the small sample compared to the multi-category DIF methods and in the large sample compared to the DSF methods. However, it was determined that the methods appeared more consistent results in a large sample. Throughout the steps, it was stated that the DIF effect was lower for the substances with DIF of the opposite signs. Therefore, if DSF analysis is not done for items that has no DIF, incorrect results may be obtained. Although, it is seen that the category combining conditions created within the scope of the study do not have a systematic effect on the results, considering choosing frequency of the combined categories differentiated the results, it is suggested that this situation should be examined in future studies.