Comparison of Passing Scores Determined by The Angoff Method in Different Item Samples
INTERNATIONAL JOURNAL OF ASSESSMENT TOOLS IN EDUCATION, cilt.7, sa.1, 2020 (ESCI, TRDizin)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 7 Sayı: 1
- Basım Tarihi: 2020
- Doi Numarası: 10.21449/ijate.699479
- Dergi Adı: INTERNATIONAL JOURNAL OF ASSESSMENT TOOLS IN EDUCATION
- Derginin Tarandığı İndeksler: Emerging Sources Citation Index (ESCI), Directory of Open Access Journals, TR DİZİN (ULAKBİM)
- Anahtar Kelimeler: Standard-setting, Angoff, Random sampling methods, Minimum passing scores, STANDARD, SUBSETS, VALIDITY
- Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
- Hacettepe Üniversitesi Adresli: Evet
Özet
In this study, the efficiency of various random sampling methods to reduce the number of items rated by judges in an Angoff standard-setting study was examined and the methods were compared with each other. Firstly, the full-length test was formed by combining Placement Test 2012 and 2013 mathematics subsets. After then, simple random sampling (SRS), content stratified (C-SRS), item-difficulty stratified (D-SRS) and content-by-difficulty random sampling (CD-SRS) methods were used to constitute different length of subsets (30%, 40%, 50%, 70%) from the full-test. In total, 16 different study conditions (4 methods x 4 subsets) were investigated. In data analysis part, ANOVA analysis was conducted to examine whether minimum passing scores (MPSs) for the subsets were significantly different from the MPSs of the full-length test. As a follow-up analysis, RMSE and SEE (Standard Error of Estimation) values were calculated for each study condition. Results indicated that the estimated Angoff MPSs were significantly different from the full-test Angoff MPS (45.12) only in the study conditions of 30%-C-SRS, 40% C-SRS, 30% D-SRS and 30%-CD-SRS. According to RMSE values, the C-SRS method had the smallest error while the SRS method had the biggest one. Moreover, SEE examinations revealed that to achieve estimations similar to the full-test Angoff MPS (within one SEE), it is sufficient to get 50% of items with the C-SRS method. C-SRS method was the more effective one compared to the others in reducing the number of items rated by judges in MPS setting studies conducted with the Angoff method.