Comparison of Often Used Analysis Methods for Rank-Ordered Data

Süleyman Demir

doi:https://doi.org/10.29329/ijpe.2023.603.1

Original article | International Journal of Progressive Education 2023, Vol. 19(5) 1-13

Comparison of Often Used Analysis Methods for Rank-Ordered Data

Süleyman Demir

pp. 1 - 13 | DOI: https://doi.org/10.29329/ijpe.2023.603.1 | Manu. Number: MANU-2303-13-0008.R1

Published online: October 16, 2023 | Number of Views: 90 | Number of Download: 288

Save PDF

Abstract

This study aims to compare the findings obtained from Rank-Ordered Judgments Scaling (ROJS), Placket-Luce Model (PLM), and Many Facet Rasch Model (MFRM) methods based on ranking judgments, which are often used in the analysis of rank-ordered data. For this purpose, one hundred senior students studying at the Faculty of Education and Faculty of Theology of Sakarya University were asked to rank pedagogical formation courses from the course they thought would be the most useful in their professional lives to the course they thought would be the least useful. The obtained data were analyzed using ROJS, PLM, and MFRM methods. When the obtained data were analyzed according to the ROJS, PLM, and MFRM, it was found that the course considered the least useful and the least preferred was the Instructional Technologies course. According to the raters, it was found that the most preferred and the most useful courses were Teaching Practice (I and II) in MFRM and ROJS, while in PLM, it was found to be the Classroom Management course. All other courses except the first-ranked course were sorted similarly in all models; the scale values in ROJS, logit values in MFRM, and worth in PLM were similar.

Keywords: Rank Ordered Data, Many Facet Rasch Model, Placket Luce Model, Judgment Scaling

How to Cite this Article?

APA 6th edition
Demir, S. (2023). Comparison of Often Used Analysis Methods for Rank-Ordered Data . International Journal of Progressive Education, 19(5), 1-13. doi: 10.29329/ijpe.2023.603.1

Harvard
Demir, S. (2023). Comparison of Often Used Analysis Methods for Rank-Ordered Data . International Journal of Progressive Education, 19(5), pp. 1-13.

Chicago 16th edition
Demir, Suleyman (2023). "Comparison of Often Used Analysis Methods for Rank-Ordered Data ". International Journal of Progressive Education 19 (5):1-13. doi:10.29329/ijpe.2023.603.1.

References

Albayrak-Sarı, A. & Gelbal, S. (2015). İkili karşılaştırmalar yargılarına ve sıralama yargılarına dayalı ölçekleme yaklaşımlarının karşılaştırılması [A comparison of scaling procedures based on pair-wise comparison and rank-order judgments scaling]. Journal of Measurement and Evaluation in Education and Psychology, 6(1), 126-141. http://dx.doi.org/10.21031/epod.30288. [Google Scholar]
Altun, A. & Gelbal S. (2014). Öğretmenlerinin kullandıkları ölçme ve değerlendirme yöntem veya araçlarının ikili karşılaştırma yöntemiyle belirlenmesi [Determining teachers’ measurement tools or techniques via pair-wise comparison method]. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 2(2), 200-209. https://doi.org/10.21031/epod.09064. [Google Scholar] [Crossref]
Anıl, D. & Güler, N. (2006). İkili karşılaştırma yöntemi ile ölçekleme çalışmasına bir örnek [An example of the scaling study by pair-wise comparison method]. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 30, 30-36. http://efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/722-published.pdf. [Google Scholar]
Bal, Ö. (2011). Seviye Belirleme Sınavı (SBS) başarısında etkili olduğu düşünülen faktörlerin sıralama yargıları kanunuyla ölçeklenmesi. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 2(2), 200-209. https://dergipark.org.tr/tr/download/article-file/65975. [Google Scholar]
Ekinci, A., Bindak, R. & Yıldırım, C. (2012). İlköğretim okulu yöneticilerinin öğretmenlerin mesleki sorunlarına empatik yaklaşımlarının ikili karşılaştırmalar metodu ile incelenmesi [A research regarding the empathic approaches of school managers about professional problems of teachers by pairwise comparisons method]. Gaziantep Üniversitesi Sosyal Bilimler Dergisi. 11(3), 759-776. https://dergipark.org.tr/tr/download/article-file/223316. [Google Scholar]
Farrokhi, F., & Esfandiari, R. (2011). A many-facet Rasch model to detect halo effect in three types of raters. Theory and Practice in Language Studies, 1(11), 1531–1540. http://dx.doi.org/10.4304/tpls.1.11.1531-1540 [Google Scholar]
Finch, H.. (2022). An introduction to the analysis of ranked response data. Practical Assessment, Research & Evaluation, 27(7), 1-20. https://scholarworks.umass.edu/pare/vol27/iss1/7/ [Google Scholar]
Güler, N., İlhan, M. & Taşdelen-Teker, G. (2018). İkili karşılaştırmalarla ölçekleme yöntemi ile Rasch analizinden elde edilen ölçek değerlerinin karşılaştırılması [Comparing the scale values obtained from pairwise comparison scaling method and rasch analysis]. İnönü Üniversitesi Eğitim Fakültesi Dergisi, 19(1), 31-48. http://dx.doi.org/10.17679/inuefd.400386. [Google Scholar]
İlhan, M. (2016). A comparison of the results of many-facet Rasch analyses based on crossed and judge pair designs. Educational Sciences: Theory & Practice, 16(2), 579-601. http://dx.doi.org/10.12738/estp.2016.2.0390. [Google Scholar]
Kan, A. (2008). Yargıcı kararlarına dayalı ölçekleme yöntemlerinin karşılaştırılması üzerine ampirik bir çalışma [A comparision of scaling methods based on judge decisions: an empirical study]. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 35, 186-194. http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/552-published.pdf. [Google Scholar]
Kijewska, K., França, J. G. C. B., de Oliveira, L. K., & Iwan, S. (2022). Evaluation of urban mobility problems and freight solutions from residents’ perspectives: a comparison of Belo Horizonte (Brazil) and Szczecin (Poland). Energies, 15(3), 710. https://doi.org/10.3390/en15030710. [Google Scholar] [Crossref]
Knoch, U., &, McNamara, T. (2015). Rasch analysis. In L. Plonsky, (Ed.), Advancing quantitative methods in second language research (pp. 275–304). Routledge. [Google Scholar]
Koçak, D. & Çokluk-Bökeoglu, Ö. (2021). Üniversite Tercih Nedenlerinin İkili Karşılaştırma ve Sıralama Yargıları Yöntemleri ile Ölçeklenmesi [Scaling university preferences with pair-wise comparison method and rank-order judgments method]. MANAS Sosyal Araştırmalar Dergisi, 10(3), 1580-1591 . DOI: https://doi.org/10.33206/mjss.874504 [Google Scholar] [Crossref]
Linacre, J. M. (1994). Many-Facet Rasch Measurement. Chicago: Mesa Press. [Google Scholar]
Linacre J. M. (2006). Rasch Analysis of Rank-Ordered Data. Journal of Applied Measurement, 7(1), 129-136. https://pubmed.ncbi.nlm.nih.gov/16385155/. [Google Scholar]
Linacre, J. M. (2014). A user’s guide to FACETS Rasch-model computer programs. http://www.winsteps.com/a/facets-manual.pdf [Google Scholar]
Özer, Y. & Acar, M. (2011). Öğretmenlik mesleği genel yeterlikleri üzerine ikili karşılaştırma yöntemiyle bir ölçekleme çalışması. Çukurova Üniversitesi Eğitim Fakültesi Dergisi, 3(40), 89-101. [Google Scholar]
Robitzsch, A., Kiefer, T., & Wu, M. (2022). TAM: Test Analysis Modules. R package version 4.1-4, https://CRAN.R-project.org/package=TAM. [Google Scholar]
Sarıdaş, G. & Nayir, F. (2021). Kültürel değerlere duyarlı öğretmen özelliklerinin sıralama yargılarıyla ölçeklenmesi [Scaling Culturally responsive teacher characteristics with ranking judgments]. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 53, 355- 377. https://doi.org/10.9779.pauefd.827009. [Google Scholar] [Crossref]
Shakir, M. K. M., Brooks, D. I., McAninch, E. A., Fonseca, T.L., Mai, V. Q., Bianco, A. C. & Thanh, D. H. (2021). Comparative effectiveness of levothyroxine, desiccated thyroid extract, and levothyroxine+liothyronine in hypothyroidism. The Journal of Clinical Endocrinology & Metabolism, 106(11). https://doi.org/10.1210/clinem/dgab478. [Google Scholar] [Crossref]
Turner, H., Kosmidis, I., Firth, D., & van Etten, J. (2021). Modelling rankings in R: The Plackettluce package. Computational Statistics, 35, 1027-1057. https://doi.org/10.1007/s00180-020-00959-3. [Google Scholar] [Crossref]

Volume 19 Issue 5

October 2023

All Articles