Investigating Invariant Item Ordering Using Mokken Scale Analysis for Dichotomously Scored Items

Ezgi Mor Dirlik

doi:https://doi.org/10.29329/ijpe.2020.248.6

Original article | International Journal of Progressive Education 2020, Vol. 16(3) 84-96

Investigating Invariant Item Ordering Using Mokken Scale Analysis for Dichotomously Scored Items

Ezgi Mor Dirlik

pp. 84 - 96 | DOI: https://doi.org/10.29329/ijpe.2020.248.6 | Manu. Number: MANU-1908-20-0005.R1

Published online: June 05, 2020 | Number of Views: 244 | Number of Download: 669

Save PDF

Abstract

Mokken models have recently started to become the preferred method of researchers from different fields in studies of nonparametric item response theory (NIRT). Despite increasing application of these models, some features of this type of modelling need further study and explanation. Invariant item ordering (IIO) is one of these areas, which the present study aims to exemplify using a real dataset and comparing the findings of different methods. The main purpose of this study is to check the IIO assumption for a large scale test by using different methods. Data relating to the high school placement test (applied in 2016) was investigated. The sample size was determined as being 250, which complies with NIRT. Two different methods have been used for dichotomous items in the IIO testing: rest-scores and P-matrix methods. The HT coefficients were also calculated in order to define the placement of item response functions. Findings show that the test battery is not suitable for Mokken scaling. IIO property was not met for any of the tests, and findings from the different methods were not consistent with each other. As for the results, the rest-score method defined more items violating IIO properties than the P-matrix method for all the tests. The HT coefficients were also estimated from the critical value, which shows that the tests do not have IIO properties. The conflicting results indicate that there is a need for new studies to investigate IIO empirically..

Keywords: Invariant Item Ordering, Nonparametric Item Response Theory, Mokken Models

How to Cite this Article?

APA 6th edition
Dirlik, E.M. (2020). Investigating Invariant Item Ordering Using Mokken Scale Analysis for Dichotomously Scored Items . International Journal of Progressive Education, 16(3), 84-96. doi: 10.29329/ijpe.2020.248.6

Harvard
Dirlik, E. (2020). Investigating Invariant Item Ordering Using Mokken Scale Analysis for Dichotomously Scored Items . International Journal of Progressive Education, 16(3), pp. 84-96.

Chicago 16th edition
Dirlik, Ezgi Mor (2020). "Investigating Invariant Item Ordering Using Mokken Scale Analysis for Dichotomously Scored Items ". International Journal of Progressive Education 16 (3):84-96. doi:10.29329/ijpe.2020.248.6.

References

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, New Jersey: Lawrence Erlbaum. [Google Scholar]
Junker, B. W., & Sijtsma, K. (2001). Nonparametric item response theory in action: An overview of the special issue. Applied Psychological Measurement, 25, 211–220. doi: 10. 1177/01466210122032028 [Google Scholar] [Crossref]
Koğar, H. (2018). Examining Invariant Item Ordering Using Mokken Scale Analysis for Polytomously Scored Items. Journal of Measurement and Evaluation in Education and Psychology, 9(4), 312–325. doi: 10.21031/epod.412689 [Google Scholar] [Crossref]
Ligtvoet, R. (2010). Essays on invariant item ordering. (Unpublished doctoral dissertation), Tilburg University, the Netherlands. [Google Scholar]
Ligtvoet, R., van der Ark, L. A., Bergsma, W. P., & Sijtsma, K. (2011). Polytomous latent scales for the investigation of the ordering of items. Psychometrika, 76, 200–216. doi: 10.1007/s11336-010-9199-8 [Google Scholar] [Crossref]
Ligtvoet, R., van der Ark, L. A., & Sijtsma, K. (2008). Selection of Alzheimer symptom items with manifest monotonicity and manifest invariant item ordering. New Trends in Psychometrics, 3(1), 225–234. [Google Scholar]
Ligtvoet, R., van der Ark, L. A., te Marvelde, J. M., & Sijtsma, K. (2010). Investigating an invariant item ordering for polytomously scored items. Educational and Psychological Measurement, 70(4), 578–595. doi: 10.1177/0013164409355697 [Google Scholar] [Crossref]
Meijer, R. R., & Baneke, J. J. (2004). Analyzing psychopathology items: A case for nonparametric item response theory modeling. Psychological Methods, 9, 354–368. [Google Scholar]
Meijer, R. R., Egberink, I. J. L., Emons, W. H. M., & Sijtsma, K. (2008). Detection and validation of unscalable item score patterns using item response theory: An illustration with Harter’s Self-Perception Profile for Children. Journal of Personality Assessment, 90, 227–238. [Google Scholar]
Meijer, R. R., & Egberink, J. L. (2012). Investigating Invariant Item Ordering in Personality and Clinical Scales: Some Empirical Findings and a Discussion. Educational and Psychological Measurement, 72(4), 589–607. [Google Scholar]
Meijer, R. R., Sijtsma, K., & Smid, N. G. (1990). Theoretical and empirical comparison of the Mokken and the Rasch approach to IRT. Applied Psychological Measurement, 14, 283–298. [Google Scholar]
Mokken, R. J. (1971). A theory and procedure of scale analysis. The Hague, Netherlands: Mouton. [Google Scholar]
Mokken, R. J. (1997). Nonparametric models for dichotomous responses. In van der Linden, W. J., & Hambleton, R. K. (Eds.), Handbook of modern item response theory (pp. 351–367). New York, NY: Springer. [Google Scholar]
Molenaar, I. W. (1997). Nonparametric models for polytomous responses. In van der Linden, W. J., & Hambleton, R. K. (Eds.), Handbook of modern item response theory (pp. 369–380). New York, NY: Springer. [Google Scholar]
Molenaar, I. W., & Sijtsma, K. (2000). MSP5 for windows. User’s manual. Groningen, Netherlands: ProGAMMA. [Google Scholar]
Dirlik, M. E. (2014). Ölçek geliştirme konulu doktora tezlerinin test ve ölçek geliştirme standartlarına uygunluğunun incelenmesi. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 5(2), 62–78. [Google Scholar]
Dirlik, M. E. & Koç, N. (2013). Eğitim kurumlarında kullanılan psikolojik testlerin ölçme standartlarına göre incelenmesi. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 8(4), 453–468. [Google Scholar]
Roorda, L. D., Houwink, A., Smits, W., Molenaar, I. W., & Geurts, A. C. (2011). Measuring upper limb capacity in poststroke patients: Development, fit of the monotone homogeneity model, unidimensionality, fit of the double monotonicity model, differential item functioning, internal consistency, and feasibility of the Stroke Upper Limb Capacity Scale, SULCS. Archives of Physical Medicine and Rehabilitation, 92, 214–227. [Google Scholar]
Roorda, L. D., Roebroeck, M. E., van Tilburg, T., Molenaar, I. W., Lankhorst, G. J., & Bouter, L. M. (2005). Measuring activity limitations in walking: Development of a hierarchical scale for patients with lower-extremity disorders who live at home. Archives of Physical Medicine and Rehabilitation, 86, 2277–2283. [Google Scholar]
Sijtsma, K., & Hemker, B. T. (1998). Nonparametric polytomous IRT models for invariant item ordering, with results for parametric models. Psychometrika, 63, 183–200. [Google Scholar]
Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage. [Google Scholar]
Sijtsma, K., & Junker, B. W. (1996). A survey of theory and methods of invariant item ordering. British Journal of Mathematical and Statistical Psychology, 49(1), 79–105. doi: 10.1111/j.2044-8317.1996.tb01076.x [Google Scholar] [Crossref]
Sijtsma, K., & Meijer, R. R. (1992). A method for investigating the intersection of item response functions in Mokken’s nonparametric IRT model. Applied Psychological Measurement, 16, 149–157. doi: 10.1177/014662169201600204 [Google Scholar] [Crossref]
Sijtsma, K., Meijer, R. R., & van der Ark, L. A. (2011). Mokken scale analysis as time goes by: An update for scaling procedures. Personality and Individual Differences, 50, 31–37. doi: 10.1016/j.paid.2010.08.016 [Google Scholar] [Crossref]
Van Abswoude, A. A., van der Ark, L. A., & Sijtsma, K. (2004). A comparative study of test data dimensionality assessment procedures under nonparametric IRT models. Applied Psychological Measurement, 28(1), 3–24. doi: 10.1177/0146621603259277 [Google Scholar] [Crossref]
Van Abswoude, A. A., Vermunt, J. K., Hemker, B. T., & van der Ark, L. A. (2004). Mokken scale analysis using hierarchical clustering procedures. Applied Psychological Measurement, 28(5), 332–354. doi: 10.1177/0146621604265510 [Google Scholar] [Crossref]
Van Schuur, W. H. (2003). Mokken scale analysis: Between the Guttman scale and parametric item response theory. Political Analysis, 11, 139–163. [Google Scholar]

Volume 16 Issue 3

June 2020

All Articles