Studying Reliability of Open Ended Mathematics Items According to the Classical Test Theory and Generalizability Theory


Guler N., GELBAL S.

KURAM VE UYGULAMADA EGITIM BILIMLERI, vol.10, no.2, pp.1011-1019, 2010 (Journal Indexed in SSCI) identifier

  • Publication Type: Article / Article
  • Volume: 10 Issue: 2
  • Publication Date: 2010
  • Title of Journal : KURAM VE UYGULAMADA EGITIM BILIMLERI
  • Page Numbers: pp.1011-1019

Abstract

In this study, the Classical test theory and generalizability theory were used for determination to reliability of scores obtained from measurement tool of mathematics success. 24 open-ended mathematics question of the TIMSS-1999 was applied to 203 students in 2007-spring semester. Internal consistency of scores was found as 0.92. For determination of interrater consistency, Kendall's concordance coefficient was calculated as 0.52. Generalizability coefficient for mathematics scores was 0.92 and phi coefficient was 0.90. The variance component of raters accounted for 2.1% of the total variance. According to all results, it was seen that measurement tool of mathematics success was reliable for determination of students' mathematics success. Although there was a difference between means of four raters' scores, it was found that there was consistency of their scores.