Kâğıt-Kalem, Bilgisayar ve Tablet Ortamında Gerçekleştirilen Sınavlar İçin Ölçme Değişmezliğinin ve Öğrenci Görüşlerinin İncelenmesi

Abstract

The aim of this study is to examine the measurement invariance of the paper-pencil, computer based and tablet based exams across application environment, to investigate whether there is any difference in exam duration and to reveal student opinions about the online application environment. The study group includes 821 fifth graders receiving education in 2016-2017 school year. The validity and reliability studies of the Mathematics Achievement Test were conducted with 402 students. For determining the items forming the final application of the Mathematics (Achievement) Test, the data obtained with the pilot study was analyzed through exploratory factor analysis. Item difficulty and item discrimination indexes were estimated for every item of the test. Item selection was made in such a way that there are three items for every seven (except one) outcomes and all of the items loaded on one dimension. The items which have the highest factor loadings, item discrimination and item difficulty indexes for the related outcome were taken into consideration during the item selection. The final form of the Mathematics Achievement Test was created with 20 multiple-choice items loading on single dimension and applied to 419 students. Whether the Mathematics Achievement Test ensure measurement invariance across application environment was investigated. In the invariance analyses, multigroup confirmatory factor analysis was utilized and the measurement invariance was examined. The times spent by the students on different application environments were compared with ANOVA. Semistructured interviews were conducted with 261 students to find out the opinions of the students about online exams. The results of the study show that the Mathematics Achievement Test ensures only the configural invariance across the groups of paper-pencil, computer and tablet. Mathematics Achievement Test illustrates that it is not valid to make comparison between the scores of the students from the paper-pencil group and the computer group and also between the scores of the students from computer group and tablet group. Mathematics Achievement Test ensured all steps of the measurement invariance between the groups of paper-pencil and tablet. This finding indicates that the means, variances and covariances of the scores which the students obtained from the exams applied as paper-pencil or on tablet are comparable. It was found out that there is a significant difference among the times spent across application environments. There is perception among the students that the exams applied based on computer and tablet will affect the scores taken from the examinations and the exam duration. Moreover it was revealed that the students want the future examinations to be applied as computer or tablet based.