Examining Measurement Invariance in Bayesian Item Response Theory Models: A Simulation Study


AYVALLI M., KELECİOĞLU H.

Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, cilt.14, sa.1, ss.19-32, 2023 (Scopus) identifier identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 14 Sayı: 1
  • Basım Tarihi: 2023
  • Doi Numarası: 10.21031/epod.1101457
  • Dergi Adı: Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi
  • Derginin Tarandığı İndeksler: Scopus, TR DİZİN (ULAKBİM)
  • Sayfa Sayıları: ss.19-32
  • Anahtar Kelimeler: Measurement invariance, bayesian IRT models, bayes factor, random item effects modelling
  • Hacettepe Üniversitesi Adresli: Evet

Özet

The aim of the study is to determine a measurement invariance cut-off point based on item parameter differences in Bayesian Item Response Theory Models. Within this scope, the Bayes factor is estimated for testing measurement invariance. For this purpose, a simulation study is conducted. The data were generated in the R software for each simulation condition under the one-parameter logistic model for 10 binary (1-0 scored) items. The invariance test was performed for various group sizes (n=500, 1000, 1500 and 2000) and difficulty parameters (dk=0, dk=0.1, dk=0.3, dk=0.5 and dk=0.7). The Bayesian analyzes were performed on the WINBUGS using the codes written in the R. A Bayes factor that provides evidence for measurement invariance was calculated depending on the parameter differences. The Savage–Dickey density ratio, one of the MCMC sampling schemas, was used to calculate the Bayes factor. As a result, if the item parameter difference is dk=0.3 and group sizes are 1500 or larger, the measurement invariance cannot be achieved. However, for small sample sizes (n=1000 or less) measurement invariance interpretation should be done carefully. When the dk=0.5, there are invariant items only in n=500. According to Bayes factor test results, evidence has been produced when dk=0.7 that measurement invariance cannot be achieved.