International Association of Educators   |  ISSN: 2834-7919   |  e-ISSN: 1554-5210
International Journal of Progressive Education
International Association of Educators

Original article | International Journal of Progressive Education 2021, Vol. 17(2) 369-384

An Approach for Being Able to Use the Options of Calculating Inter-coder Reliability Manually and Through Software in Qualitative Research of Education and Training in Sports

Ali Sevilmiş, Ozer Yildiz

pp. 369 - 384   |  DOI: https://doi.org/10.29329/ijpe.2021.332.23   |  Manu. Number: MANU-2009-30-0005.R1

Published online: April 07, 2021  |   Number of Views: 188  |  Number of Download: 815

Save PDF

Abstract

Reliability that can be proved by numeric indicators in quantitative studies has become a very discussible issue. The reason for this is to be thought that in qualitative researches, reliability is not based on positive perspective and those forming reliability criteria is difficult.  However, for testing the reliability of a qualitative study or raising it to the top level, some ways are used. The objective of this study is to introduce an approach to be able to use the options of inter-coder reliability calculating manually and through software in qualitative researches of education and training in sports. This research was conducted by document review method. In this context, how inter-coder reliability in qualitative researches will be formulated, how it will be calculated in software dimension, and the dimensions  of  inter-coder agreement, code frequency in document, code overlapping rate of segment level, and revealing code frequency were discussed. In the study, document analysis technique among qualitative research designs and in analyses content analysis method were used. The study results revealed with the examples that inter-coder reliability could be calculated in the various ways and that percentage flexibility could differ in every formula. Also, the options of inter-coder agreement were calculated by means of software, and it was identified that each option used had a different flexibility. This study, in terms of its originality, drawing the existing reliability from related to qualitative study from the framework of inter-coder agreement, suggests a synthesis of contemporary viewpoints. It is considered that in inter-coder qualitative studies, especially in sports education and training studies, will guide to the relevant researchers.

Keywords: Calculation, Inter-Coder Reliability, Inter-Coder Agreement, Maxqda, Option, Qualitative

How to Cite this Article?

APA 6th edition
Sevilmis, A. & Yildiz, O. (2021). An Approach for Being Able to Use the Options of Calculating Inter-coder Reliability Manually and Through Software in Qualitative Research of Education and Training in Sports . International Journal of Progressive Education, 17(2), 369-384. doi: 10.29329/ijpe.2021.332.23

Harvard
Sevilmis, A. and Yildiz, O. (2021). An Approach for Being Able to Use the Options of Calculating Inter-coder Reliability Manually and Through Software in Qualitative Research of Education and Training in Sports . International Journal of Progressive Education, 17(2), pp. 369-384.

Chicago 16th edition
Sevilmis, Ali and Ozer Yildiz (2021). "An Approach for Being Able to Use the Options of Calculating Inter-coder Reliability Manually and Through Software in Qualitative Research of Education and Training in Sports ". International Journal of Progressive Education 17 (2):369-384. doi:10.29329/ijpe.2021.332.23.
References
  1. Allan, G. (2020). Qualitative research. In Handbook for research students in the social sciences (pp. 177-189). Routledge.  [Google Scholar]
  2. Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement. Sociological Methods Research, 42(3), 294-320. https://doi.org/https://doi.org/10.1177/0049124113500475  [Google Scholar] [Crossref] 
  3. Cho, J., & Trent, A. (2006). Validity in qualitative research revisited. Journal Qualitative Research, 6(3), 319-340. https://doi.org/https://doi.org/10.1177/1468794106065006  [Google Scholar] [Crossref] 
  4. Davies, K. (2012). Content analysis of research articles in information systems (LIS) journals. Library Information Research, 36(112), 16-28. https://doi.org/https://doi.org/10.29173/lirg495  [Google Scholar] [Crossref] 
  5. Denzin, N. K., & Lincoln, Y. S. (2011). The Sage handbook of qualitative research. Sage.  [Google Scholar]
  6. Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry: A guide to methods. Sage.  [Google Scholar]
  7. Freelon, D. G. (2010). ReCal: Intercoder reliability calculation as a web service. International Journal of Internet Science, 5(1), 20-33.  [Google Scholar]
  8. Friese, S. (2019). Qualitative data analysis with ATLAS.ti. Sage.  [Google Scholar]
  9. Gomes, S., & Duarte, V. (2020). What about ethics? Developing qualitative research in confinement settings. European Journal of Criminology, 17(4), 461-479. https://doi.org/https://doi.org/10.1177/1477370818801305  [Google Scholar] [Crossref] 
  10. González-Prieto, Á., Perez, J., Diaz, J., & López-Fernández, D. J. a. p. a. (2020). Inter-coder agreement for improving reliability in software engineering qualitative research. 1-37. https://arxiv.org/pdf/2008.00977.pdf.  [Google Scholar]
  11. Hanson, W. E., Creswell, J. W., Clark, V. L. P., Petska, K. S., & Creswell, J. D. (2005). Mixed methods research designs in counseling psychology. Journal of Counseling Psychology, 52(2), 224-234 https://doi.org/https://doi.org/10.1037/0022-0167.52.2.224  [Google Scholar] [Crossref] 
  12. Hennink, M., Hutter, I., & Bailey, A. (2020). Qualitative research methods. Sage.  [Google Scholar]
  13. Joffe, H., & Yardley, L. (2004). Content and thematic analysis. In Research Methods for Clinical Health Psychology. (pp. 56-68). London, Sage.  [Google Scholar]
  14. Kıral, B. (2020). Document analysis as a qualitative data analysis method. Siirt University Journal of Social Science Institute. 8(15), 170-189.  [Google Scholar]
  15. Kuckartz, U. (2016). Typenbildung und typenbildende Inhaltsanalyse in der empirischen Sozialforschung. In Junge Menschen sprechen mit sterbenden Menschen (pp. 31-53). Springer.  [Google Scholar]
  16. Lavrakas, P. J. (2008). Encyclopedia of survey research methods. Sage.  [Google Scholar]
  17. Lester, J. N., Cho, Y., & Lochmiller, C. R. (2020). Learning to do qualitative data analysis: A starting point. Human Resource Development Review, 19(1), 94-106. https://doi.org/https://doi.org/10.1177/1534484320903890  [Google Scholar] [Crossref] 
  18. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills: Sage.  [Google Scholar]
  19. Lombard, M., Snyder‐Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Journal Human communication Research, 28(4), 587-604. https://doi.org/https://doi.org/10.1111/j.1468-2958.2002.tb00826.x  [Google Scholar] [Crossref] 
  20. MacPhail, C., Khoza, N., Abler, L., & Ranganathan, M. (2016). Process guidelines for establishing intercoder reliability in qualitative studies. Journal Qualitative Research, 16(2), 198-212. https://doi.org/https://doi.org/10.1177/1468794115577012  [Google Scholar] [Crossref] 
  21. MAXQDA. (2020). MAXQDA 2020 Manual. 1-736. https://www.maxqda.com/download/manuals/MAX2020-Online-Manual-Complete-EN.pdf  [Google Scholar]
  22. Maxwell, J. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62(3), 279-301. https://doi.org/https://doi.org/10.17763/haer.62.3.8323320856251826  [Google Scholar] [Crossref] 
  23. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Sage.  [Google Scholar]
  24. Nili, A., Tate, M., & Barros, A. (2017). A critical analysis of inter-coder reliability methods in information systems research. In Riemer, K, Indulska, M, & Tuunainen, V (Eds.) Proceedings of the 28th Australasian Conference on Information Systems. University of Tasmania, Australia,  1-11. [Google Scholar]
  25. Nili, A., Tate, M., Barros, A., & Johnstone, D. (2020). An approach for selecting and using a method of inter-coder reliability in information management research. International Journal of Information Management, 54, 1-13. https://doi.org/https://doi.org/10.1016/j.ijinfomgt.2020.102154  [Google Scholar] [Crossref] 
  26. O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: debates and practical guidelines. International Journal of Qualitative Methods, 19, 1-13. https://doi.org/https://doi.org/10.1177/1609406919899220  [Google Scholar] [Crossref] 
  27. Olson, J. D., McAllister, C., Grinnell, L. D., Gehrke Walters, K., & Appunn, F. (2016). Applying constant comparative method with multiple investigators and inter-coder reliability. J Qualitative Report, 21(1). 26-42. [Google Scholar]
  28. Rädiker, S., & Kuckartz, U. (2019). Analyse qualitativer Daten mit MAXQDA. Springer.  [Google Scholar]
  29. Richards, T. J., & Richards, L. (1994). Using computers in qualitative research. Handbook of qualitative research, 2, 445-462.  [Google Scholar]
  30. Rolfe, G. (2006). Validity, trustworthiness and rigour: quality and the idea of qualitative research. Journal of advanced nursing, 53(3), 304-310. https://doi.org/https://doi.org/10.1111/j.1365-2648.2006.03727.x  [Google Scholar] [Crossref] 
  31. Sandelowski, M. (1993). Rigor or rigor mortis: the problem of rigor in qualitative research. J Advances in Nursing Science, 16(2), 1-8. https://doi.org/10.1097/00012272-199312000-00002  [Google Scholar] [Crossref] 
  32. Venkatesh, V., Brown, S. A., & Bala, H. (2013). Bridging the qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. Journal MIS Quarterly, 37 (1), 21-54.  [Google Scholar]
  33. Weitzman, E., & Miles, M. B. (1995). Computer programs for qualitative data analysis. Sage.  [Google Scholar]
  34. Wilson-Lopez, A., Angela Minichiello, P., & Green, T. (2019). An inquiry into the use of intercoder reliability measures in qualitative research. ASEE Annual Conference proceedings, 1-14. file:///C:/Users/Windows%2010/Downloads/an-inquiry-into-the-use-of-intercoder-reliability-measures-in-qualitative-research%20(1).pdf [Google Scholar]
  35. Yıldırım, A. (1999). Nitel arastirma yontemlerinin temel ozellikleri ve egitim arastirmalarindaki yeri ve onemi [Basic features of qualitative research methods and their place and importance in educational research]. Education and Science, 23(112), 7-17.  [Google Scholar]
Download
Metric
History
Related

Volume 17 Issue 2

April 2021

All Articles

International Journal of Progressive Education

International Association of Educators.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License.
Privacy policy     |     Terms of Use     |     Use of Cookies
Creative Commons Lisansı
© 2012 - 2024 - THDSoft e-Journal Management System.