When Engagement Is Mistaken for Critical Reading: A Construct Audit of Technology-Supported EFL Research

Authors

DOI:

https://doi.org/10.24903/bej.v6i2.2500

Keywords:

critical reading, educational technology, EFL, construct audit, measurement, digital literacy

Abstract

Technology-supported EFL studies often report engagement, motivation, usefulness, and perceived learning gains. These outcomes matter, but they do not necessarily show that students can evaluate evidence, question sources, detect bias, or defend a judgment. This study audits the constructs measured in five project manuscripts involving Blooket, Quizizz, Canva, Kahoot!, and Wayground. The corpus represents 133 participant-records across manuscripts, which were not pooled because the samples, instruments, and contexts differ. A comparative secondary construct audit classified each manuscript against seven dimensions: technology acceptance, engagement, language processing, feedback and self-monitoring, social-dialogic interaction, multimodal meaning-making, and epistemic critical reading. Engagement was directly represented in all five manuscripts. Acceptance, language processing, and feedback were also common. Social-dialogic interaction appeared directly in only one manuscript, multimodal meaning-making in one, and none directly measured source credibility, evidence quality, bias, counterargument, or justified critical judgment. The pattern suggests a measurement gap rather than a failure of the technologies themselves. Positive learner experience is best treated as an enabling condition, not a proxy for critical reading. The study proposes a Critical Reading Technology Measurement Framework that separates acceptance and engagement from comprehension, monitoring, dialogue, multimodal interpretation, and epistemic evaluation. Future technology-supported EFL research should pair perception measures with performance tasks that make critical-reading operations observable and assessable

References

Algouzi, S., Alzubi, A. A. F., & Nazim, M. (2023). Enhancing EFL students' critical thinking skills using a technology-mediated self-study approach: EFL graduates and labor market in perspective. PLOS ONE, 18(10), e0293273. https://doi.org/10.1371/journal.pone.0293273

Anggia, H., & Habók, A. (2024). University students' metacognitive awareness of reading strategies (MARS) in online reading and MARS' role in their English reading comprehension. PLOS ONE, 19(11), e0313254. https://doi.org/10.1371/journal.pone.0313254

Bergdahl, N., Bond, M., Sjöberg, J., Dougherty, M., & Oxley, E. (2024). Unpacking student engagement in higher education learning analytics: A systematic review. International Journal of Educational Technology in Higher Education, 21, 63. https://doi.org/10.1186/s41239-024-00493-y

Chan, S., & Lo, N. (2024). Enhancing EFL/ESL instruction through gamification: A comprehensive review of empirical evidence. Frontiers in Education, 9, 1395155. https://doi.org/10.3389/feduc.2024.1395155

Cigdem, H., Ozturk, M., Karabacak, Y., Atik, N., Gürkan, S., & Aldemir, M. H. (2024). Unlocking student engagement and achievement: The impact of leaderboard gamification in online formative assessment for engineering education. Education and Information Technologies, 29, 24835–24860. https://doi.org/10.1007/s10639-024-12845-2

Dori, Y. J., & Lavi, R. (2023). Teaching and assessing thinking skills and applying educational technologies in higher education. Journal of Science Education and Technology, 32, 773–777. https://doi.org/10.1007/s10956-023-10072-x

Drugova, E., Zhuravleva, I., Zakharova, U., & Latipov, A. (2024). Learning analytics driven improvements in learning design in higher education: A systematic literature review. Journal of Computer Assisted Learning, 40(2), 510–524. https://doi.org/10.1111/jcal.12894

Galindo-Domínguez, H., Bezanilla, M.-J., Campo, L., Fernández-Nogueira, D., & Poblete, M. (2023). A teachers' based approach to assessing the perception of critical thinking in Education university students based on their age and gender. Frontiers in Education, 8, 1127705. https://doi.org/10.3389/feduc.2023.1127705

González-Mujico, F. de L. (2024). Measuring student and educator digital competence beyond self-assessment: Developing and validating two rubric-based frameworks. Education and Information Technologies, 29, 13299–13324. https://doi.org/10.1007/s10639-023-12363-7

Jiang, L. (George), & Hafner, C. (2025). Digital multimodal composing in L2 classrooms: A research agenda. Language Teaching, 58(4), 528–546. https://doi.org/10.1017/S0261444824000107

Jiang, L., Li, Z., & Leung, J. S. C. (2024). Digital multimodal composing as translanguaging assessment in CLIL classrooms. Learning and Instruction, 92, 101900. https://doi.org/10.1016/j.learninstruc.2024.101900

Kiili, C., Räikkönen, E., Bråten, I., Strømsø, H. I., & Hagerman, M. S. (2023). Examining the structure of credibility evaluation when sixth graders read online texts. Journal of Computer Assisted Learning, 39(3), 954–969. https://doi.org/10.1111/jcal.12779

Lam, K. Y., & Putri, E. W. (2024). A study on the affordances of digital fairy-tale rewriting: A critical literacy approach to digital multimodal composing. System, 127, 103536. https://doi.org/10.1016/j.system.2024.103536

Li, M., Ma, S., & Shi, Y. (2023). Examining the effectiveness of gamification as a tool promoting teaching and learning in educational settings: A meta-analysis. Frontiers in Psychology, 14, 1253549. https://doi.org/10.3389/fpsyg.2023.1253549

Mejías-Acosta, A., D'Armas Regnault, M., Vargas-Cano, E., Cárdenas-Cobo, J., & Vidal-Silva, C. (2024). Assessment of digital competencies in higher education students: Development and validation of a measurement scale. Frontiers in Education, 9, 1497376. https://doi.org/10.3389/feduc.2024.1497376

Orhan, A. (2023). Fake news detection on social media: The predictive role of university students' critical thinking dispositions and new media literacy. Smart Learning Environments, 10, 29. https://doi.org/10.1186/s40561-023-00248-8

Otto, S., Bertel, L. B., Lyngdorf, N. E. R., Markman, A. O., Andersen, T., & Ryberg, T. (2024). Emerging digital practices supporting student-centered learning environments in higher education: A review of literature and lessons learned from the COVID-19 pandemic. Education and Information Technologies, 29, 1673–1696. https://doi.org/10.1007/s10639-023-11789-3

Parmigiani, D., Nicchia, E., Murgia, E., & Ingersoll, M. (2024). Formative assessment in higher education: An exploratory study within programs for professionals in education. Frontiers in Education, 9, 1366215. https://doi.org/10.3389/feduc.2024.1366215

Paulsen, L., & Lindsay, E. (2024). Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic review. Education and Information Technologies, 29, 14279–14308. https://doi.org/10.1007/s10639-023-12401-4

Pegalajar Palomino, M. del C., & Rodríguez Torres, Á. F. (2023). Digital literacy in university students of education degrees in Ecuador. Frontiers in Education, 8, 1299059. https://doi.org/10.3389/feduc.2023.1299059

Sanabria-Z, J., Alfaro-Ponce, B., González-Pérez, L. I., & Ramírez-Montoya, M. S. (2024). Reusable educational resources for developing complex thinking on open platforms. Education and Information Technologies, 29, 1173–1199. https://doi.org/10.1007/s10639-023-12316-0

Schenck, A. (2024). Examining relationships between technology and critical thinking: A study of South Korean EFL learners. Education Sciences, 14(6), 652. https://doi.org/10.3390/educsci14060652

Vergili, M., & Kara, M. (2024). An investigation of students and teachers' new media literacy: The contributing characteristics with the moderator role of gender. Research and Practice in Technology Enhanced Learning, 19, 029. https://doi.org/10.58459/rptel.2024.19029

Wong, J. T., & Hughes, B. S. (2023). Leveraging learning experience design: Digital media approaches to influence motivational traits that support student learning behaviors in undergraduate online courses. Journal of Computing in Higher Education, 35, 595–632. https://doi.org/10.1007/s12528-022-09342-1

Zhang, Z., & Crawford, J. (2024). EFL learners' motivation in a gamified formative assessment: The case of Quizizz. Education and Information Technologies, 29, 6217–6239. https://doi.org/10.1007/s10639-023-12034-7

Downloads

Published

2024-08-28

How to Cite

Nur, D. R. (2024). When Engagement Is Mistaken for Critical Reading: A Construct Audit of Technology-Supported EFL Research. Borneo Educational Journal (Borju), 6(2), 340–354. https://doi.org/10.24903/bej.v6i2.2500

Issue

Vol. 6 No. 2 (2024): August

Section

Articles

License

Copyright (c) 2025 Dedi Rahman Nur

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.