International Association of Educators   |  ISSN: 2834-7919   |  e-ISSN: 1554-5210
International Journal of Progressive Education
International Association of Educators

Original article | International Journal of Progressive Education 2021, Vol. 17(2) 194-209

Modelling of the Attitude-Achievement Paradox in TIMSS 2015 with respect to the Extreme Response Style Using Multidimensional Item Response Theory

Munevver Ilgun Dibek & Rahime Nukhet Cıkrıkcı

pp. 194 - 209   |  DOI: https://doi.org/10.29329/ijpe.2021.332.12   |  Manu. Number: MANU-2008-12-0003

Published online: April 07, 2021  |   Number of Views: 206  |  Number of Download: 694

Save PDF

Abstract

This study aims to first investigate the effect of the extreme response style (ERS) which could lead to an attitude-achievement paradox among the countries participating in the Trends in International Mathematics and Science Study (TIMSS 2015), and then to determine the individual- and country-level relationships between attitude and achievement by adjusting the effect of ERS. For the sample of this correlational study, 500 students were randomly selected from each of the 15 countries that participated in TIMSS 2015. The differences in the ERS tendency of the countries were determined by performing MANOVA. To determine the effect of ERS, two different multidimensional item response theory (MIRT) models were used: one did not include the ERS trait as a dimension while the other included this trait as a dimension. The results were analyzed with Latent GOLD 5.1 and WinBUGS software. To determine the relationship between attitudinal variables and achievement, the correlation values based on the observed scores and MIRT models were obtained. Whether there was any significant difference between these correlation values was determined by Fisher's rz transformation. The findings of this study were as follows: (a) the model in which the ERS trait was included as a dimension best fit the data and (b) the correlation values based on the observed scores were negative and those based on the MIRT models were positive, with the two statistically differing from each other. ERS is one of the factors causing the achievement-attitude paradox; however, it not sufficient to explain this paradox.

Keywords: Multidimensional Item Response Theory, TIMSS, Attitude-Achievement Paradox, Extreme Response Style

How to Cite this Article?

APA 6th edition
Dibek, M.I. & Cikrikci, R.N. (2021). Modelling of the Attitude-Achievement Paradox in TIMSS 2015 with respect to the Extreme Response Style Using Multidimensional Item Response Theory . International Journal of Progressive Education, 17(2), 194-209. doi: 10.29329/ijpe.2021.332.12

Harvard
Dibek, M. and Cikrikci, R. (2021). Modelling of the Attitude-Achievement Paradox in TIMSS 2015 with respect to the Extreme Response Style Using Multidimensional Item Response Theory . International Journal of Progressive Education, 17(2), pp. 194-209.

Chicago 16th edition
Dibek, Munevver Ilgun and Rahime Nukhet Cikrikci (2021). "Modelling of the Attitude-Achievement Paradox in TIMSS 2015 with respect to the Extreme Response Style Using Multidimensional Item Response Theory ". International Journal of Progressive Education 17 (2):194-209. doi:10.29329/ijpe.2021.332.12.
References
  1. Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64, 359-373. [Google Scholar]
  2. Bachman, J. G., O’Malley, P. M., & Freedman-Doan, P. (2010). Response styles revisited:Racial/ethnic and gender differences in extreme responding (Monitoring the Future Occasional Paper No. 72). Ann Arbor, MI: Institute for Social Research.  [Google Scholar]
  3. Bandura, A. (1994). Self-efficacy. Encyclopedia of Human Behavior, 4, 71–81. [Google Scholar]
  4. Baumgartner, H. and Steenkamp, J. E.M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143-156. [Google Scholar]
  5. Billiet, J. B., and McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7, 608-628. [Google Scholar]
  6. Bolt, D. M., and Johnson, T. R. (2009). Addressing score bias and differential item functioning due to individual differences in response style. Applied Psychological Measurement, 33,335-352. [Google Scholar]
  7. Bolt, D. M., and Newton, J. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814-833. [Google Scholar]
  8. Buckley, J. (2009). Cross-national response styles in international educational assessment: Evidence from PISA 2006. NCES Conference on the Program for International Student Assessment: What we can learn from. Retrieved from https://edsurvey.rti.org/PISA/  [Google Scholar]
  9. Cheung, G. W., and Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross cultural research using structural equation modeling. Journal of Cross- Cultural Psychology, 31, 187-212. [Google Scholar]
  10. Cronbach, L. J. (1946). Response sets and test validity. Educational and Psychological Measurement, 6, 475-494. [Google Scholar]
  11. De Jong M. G., Steenkamp J.-B. E. M., Fox J.-P., Baumgartner H. (2008). Using item response theory to measure extreme response style in marketing research: A global investigation. Journal of Marketing Research, 45(1), 104-115. [Google Scholar]
  12. Engle, P. J. (2016). Response Style in the Political Survey.(Unpublished doctoral dissertation).Unıversıty of Wisconsin Madison. [Google Scholar]
  13. Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-3. [Google Scholar]
  14. Fraenkel, J.R., and Wallen, N.E. (2006). How to design and evaluate research in education. New York: McGraw-Hill. [Google Scholar]
  15. Geyer, C. J. (1992). On the convergence of Monte Carlo maximum likelihood calculations. Technical Report 571, School of Statistics, Univ. Minnesota. [Google Scholar]
  16. Greenleaf, E. A. (1992a). Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research, 29, 176- 188. [Google Scholar]
  17. Harter, S. (1999). The construction of the self: A developmental perspective. New York: Guildford Press. [Google Scholar]
  18. Harumi, C. A. (2011). Cross-cultural differences in response styles (Unpublished doctoral dissertation). Washington State University. [Google Scholar]
  19. Harzing, A. (2006). Response styles in cross-national survey research. International Journal of Cross Cultural Management, 6, 243-265. [Google Scholar]
  20. Hofstede, G. H. (2001). Cultures consequences: Comparing values, behaviors, institutions, and organizations across nations (2nd ed.). Thousand Oaks, California: Sage Publications, Inc. [Google Scholar]
  21. İlgün Dibek, M., Yavuz, H. & Çokluk Bökeoğlu, Ö. (2018).  Tutum - başarı paradoksunda tepki stillerinin rolü: dokuz ülkenin karşılaştırılması, Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 18(2), 932-952. [Google Scholar]
  22. Johnson, T.R., & Bolt, D. M. (2010). On the use of factor-analytic mutinomial logit item response models to account for individual differences in response style. Journalof Educational and Behavioral Statistics, 35, 92-114. [Google Scholar]
  23. Johnson, T., Kulesa, P., Cho, Y. I., and Shavitt, S. (2005). The relation between culture and response styles: evidence from 19 countries. Journal of Cross-Cultural Psychology, 36, 264-277. [Google Scholar]
  24. Lu, Y. (2012). A multilevel multidimensional ıtem response theory model to address the role of response style on measurement of attitudes in PISA 2006,(Dissertation).Universıty of Wisconsin-Madison. [Google Scholar]
  25. Lu, Y. and Bolt, D.M. (2015). Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style. Large-scale Assessments in Education,3(2), 1-18. doi:1.1186/s40536-015-0012-. [Google Scholar] [Crossref] 
  26. Kadijevich, D. (2008). TIMSS 2003: Relating dimensions of mathematics attitude to mathematics achievement. Zbornik instituta za Pedagogical Research, 40(2), 327–346. doi: 1.2298/ZIPI0802327K [Google Scholar] [Crossref] 
  27. Marsh, H. W., Trautwein, U., Lüdtke, O., Köller, O and Baumert, J. (2005). Academic self-concept, interest, grades and standardized test scores: Reciprocal effects models of causal ordering. Child Development, 76(2), 397-416. [Google Scholar]
  28. Marsh, H.W, Seaton, M., Trautwein, U., Ludtke, O., Hau, K.T., O’Mara, A.J., and Craven, R.G. (2008). The big fish little pond effect stands up to critical scrutiny: Implications for theory, methodology, and future research. Educational Psychology Review, 20, 319–35. [Google Scholar]
  29. Martin, M. O., Mullis, I. V. S., Hooper, M., Yin, L., Foy, P., and Palazzo, L. (2016). Creating and Interpreting the TIMSS 2015 Context Questionnaire Scales. In M. O. Martin, I. V. S. Mullis, & M. Hooper (Eds.), Methods and Procedures in TIMSS 2015 (pp. 15.1-15.312). Retrieved from Boston College, TIMSS &PIRLS International Study Center website: http://timss.bc.edu/publications/timss/2015-methods/chapter-15.html [Google Scholar]
  30. MedCalc Software bvba (2018). MedCalc Statistical Software version 18.2.1, Ostend, Belgium. Retrieved from https://www.medcalc.org. [Google Scholar]
  31. Moors, G. (2004). Facts and artifacts in the comparison of attitudes among ethnic minorities. A multilevel latent class structure model with adjustment for response style behavior. European Sociological Review, 20, 303-32. [Google Scholar]
  32. Ntzoufras, I. (2009). Bayesian modeling using WinBUGS. Wiley Series in Computational Statistics, Hoboken, USA. [Google Scholar]
  33. OECD (2017). PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving, revised edition, PISA, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264281820-en [Google Scholar]
  34. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver & L. S. Wrightman (Eds.), Measures of Personality and Social Psychological Attitudes (Vol. 1). San Diego, CA: Academic Press. [Google Scholar]
  35. Prediger, D. J. (1999). Basic structure of work-relevant abilities. Journal of Counseling Psychology, 46, 172-184. [Google Scholar]
  36. Reckase, M. (2009). Multidimensional item response theory. Dordrecht: Springer. [Google Scholar]
  37. Reynolds, N., and Smith, A. (2010). Assessing the impact of response styles on cross-cultural service quality evaluation: A simplified approach to eliminating the problem. Journal of Service Research, 13, 230–243. doi: 1.1177/1094670509360408 [Google Scholar] [Crossref] 
  38. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351-357. [Google Scholar]
  39. Rodriguez, M. C. (2004). The role of classroom assessment in student performance on TIMSS. Applied Measurement in Education, 17(1), 1-24. doi: 1.1207/s15324818ame1701_1 [Google Scholar] [Crossref] 
  40. Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (Complete samples). Biometrika, 52(3/4), 591-611. [Google Scholar]
  41. Shen,C.and Tam, H.P. (2008) The paradoxical relationship between student achievement and self-perception: a cross-national analysis based on three waves of TIMSS data, Educational Research and Evaluation, 14(1), 87-100, DOI: 1.1080/13803610801896653 [Google Scholar]
  42. Sinharay, S. (2004). Experiences with markov chain monte carlo convergence assessment in two psychometric examples. Journal of Educational and Behavioral Statistics, 29, 461– 488. [Google Scholar]
  43. Smith, P. B. (2004). Acquiescent response bias as an aspect of cultural communication style. Journal of Cross-Cultural Psychology, 35, 50-61. [Google Scholar]
  44. Spiegelhalter, D., Thomas, A., and Best, N. (2004). WinBUGS version 1.4 [Computer program]. Cambridge, UK: MRC Biostatistics Unit, Institute of Public Health. [Google Scholar]
  45. van de Gaer, E. and Adams,R.(2010, May). The Modeling of Response Style Bias: An Answer to the Attitude-Achievement Paradox?, paper presented at the annual conference of the American Educational Research Association, Denver, Colorado, USA.  [Google Scholar]
  46. Van de gaer, E., Grisay, A., Schulz, W. and Gebhardt, E. (2012). The reference group effect an explanation of the paradoxical relationship between academic achievement and self-confidence across countries. Journal of Cross-Cultural Psychology, 43(8), 1205-1228. [Google Scholar]
  47. van Rosmalen J., van Herk H., Groenen P. J. F. (2010). Identifying response styles: A latent-class bilinear multinomial logit model. Journal of Marketing Research, 47, 157-172. [Google Scholar]
  48. Vermunt, J. K., and Magidson, J. (2008). LG-Syntax User’s Guide: Manual for Latent Gold 4.5 Syntax Module. Belmont, MA: Statistical Innovations, Inc. [Google Scholar]
Download
Metric
History
Related

Volume 17 Issue 2

April 2021

All Articles

International Journal of Progressive Education

International Association of Educators.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License.
Privacy policy     |     Terms of Use     |     Use of Cookies
Creative Commons Lisansı
© 2012 - 2024 - THDSoft e-Journal Management System.