Evaluating Grammarly-Assisted Post-Editing of Google Translate Outputs in Indonesian-English Academic Texts: An Exploratory Corpus Study with Expert Assessment
DOI:
https://doi.org/10.24903/sj.v11i2.2387Keywords:
Google Translate, Grammarly, machine translation post-editing, translation quality assessment, academic translationAbstract
Background:This mixed-method study investigates the effectiveness of integrating Google Translate (GT) and Grammarly as a combined English-for-Academic-Purposes (EAP) learning and teaching intervention, aiming to improve the accuracy, clarity, and readability of Indonesian-to-English academic texts.
Methodology:
The study employs a sequential explanatory design combining quantitative error analysis with qualitative expert feedback. Purposive sampling was used to select a 7,500-word corpus of published Indonesian academic articles across six disciplines: linguistics, technology, economics, engineering, medical science, and law. Six EAP instructors/translators with advanced IELTS scores served as raters. Machine-generated translations via GT were post-edited using Grammarly. Quantitative data consisted of 519 Grammarly-detected writing issues categorized into clarity, correctness, delivery, and engagement. Qualitative data were collected through open-ended questionnaires based on Machali’s Translation Quality Assessment rubric. Descriptive statistics and thematic coding were used for analysis.
Findings:
Grammarly post-editing reduced grammatical and stylistic errors by over 54% in correctness and 35% in clarity categories. However, human intervention remained essential for addressing semantic nuance and cultural appropriateness. Raters evaluated engineering and technology texts more favorably (mean score = 5.8/7) than linguistics texts (mean = 4.3/7), indicating domain-specific variation in translation quality.
Conclusion:The integration of GT and Grammarly improves the overall quality of machine-translated academic texts, particularly in grammatical accuracy and clarity. Nevertheless, expert human involvement is still required to ensure semantic precision and contextual appropriateness for publication-ready outputs.
Originality:This study introduces a practical way to combine Google Translate and Grammarly for EAP learning, while showing that human expertise is still essential for high-quality academic writing.
References
Abu Qub’a, A., Abu Guba, M. N., & Fareh, S. (2024). Exploring the use of Grammarly in assessing English academic writing. Heliyon, 10(15), Article e34893. https://doi.org/10.1016/j.heliyon.2024.e34893
Barrot, J. S. (2023). Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy. Computer Assisted Language Learning, 36(4), 584-607. https://doi.org/10.1080/09588221.2021.1936071
Bassnett, S. (2002). Translation studies (3rd ed.). Routledge. https://doi.org/10.4324/9780203427460
Bentivogli, L., Bisazza, A., Cettolo, M., & Federico, M. (2016). Neural versus phrase-based machine translation quality: A case study. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 257-267). Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1025
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Castilho, S., Moorkens, J., Gaspari, F., Calixto, I., Tinsley, J., & Way, A. (2017). Is neural machine translation the new state of the art? The Prague Bulletin of Mathematical Linguistics, 108, 109-120. https://doi.org/10.1515/pralin-2017-0013
Chamoun, E., Schlichtkrull, M., & Vlachos, A. (2024). Automated focused feedback generation for scientific writing assistance. In Findings of the Association for Computational Linguistics: ACL 2024 (pp. 9742-9763). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-acl.580
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE Publications. https://us.sagepub.com/en-us/nam/product/978-1483344379
Dembsey, J. M. (2017). Closing the Grammarly gaps: A study of claims and feedback from an online grammar program. The Writing Center Journal, 36(1), 63-100. https://www.jstor.org/stable/44252638
Dizon, G., & Gayed, J. M. (2024). A systematic review of Grammarly in L2 English writing contexts. Cogent Education, 11(1), Article 2397882. https://doi.org/10.1080/2331186X.2024.2397882
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6 Pt 2), 2134-2156. https://doi.org/10.1111/1475-6773.12117
Fiederer, R., & O’Brien, S. (2009). Quality and machine translation: A realistic objective? The Journal of Specialised Translation, 11, 52-74. https://www.jostrans.org/issue11/art_fiederer_obrien.php
Freitag, M., Foster, G., Grangier, D., Ratnakar, V., Tan, Q., & Macherey, W. (2021). Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9, 1460-1474. https://doi.org/10.1162/tacl_a_00437
Godwin-Jones, R. (2022). Partnering with AI: Intelligent writing assistance and instructed language learning. Language Learning & Technology, 26(2), 5-24. https://doi.org/10.64152/10125/73474
Guerreiro, N. M., Alves, D. M., Waldendorf, J., Haddow, B., Birch, A., Colombo, P., & Martins, A. F. T. (2023). Hallucinations in large multilingual translation models. Transactions of the Association for Computational Linguistics, 11, 1500-1517. https://doi.org/10.1162/tacl_a_00615
Guo, Q., Feng, R., & Hua, Y. (2022). How effectively can EFL students use automated written corrective feedback (AWCF) in research writing? Computer Assisted Language Learning, 35(9), 2312-2331. https://doi.org/10.1080/09588221.2021.1879161
Kocmi, T., & Federmann, C. (2023). Large language models are state-of-the-art evaluators of translation quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation (pp. 193-203). European Association for Machine Translation. https://aclanthology.org/2023.eamt-1.19/
Koehn, P., & Knowles, R. (2017). Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation (pp. 28-39). Association for Computational Linguistics. https://doi.org/10.18653/v1/W17-3204
Koltovskaia, S. (2020). Student engagement with automated written corrective feedback provided by Grammarly: A multiple case study. Assessing Writing, 44, Article 100450. https://doi.org/10.1016/j.asw.2020.100450
Koltovskaia, S. (2023). Postsecondary L2 writing teachers’ use and perceptions of Grammarly as a complement to their feedback. ReCALL, 35(3), 290-304. https://doi.org/10.1017/S0958344022000179
Lesznyák, M., Bakti, M., & Sermann, E. (2024). Reading takes it all? The role of language competence and subject knowledge in legal translation. The Interpreter and Translator Trainer, 18(2), 192-211. https://doi.org/10.1080/1750399X.2024.2344177
Machali, R. (2009). Pedoman bagi penerjemah: Panduan lengkap bagi Anda yang ingin menjadi penerjemah profesional. Kaifa. https://opac.perpusnas.go.id/DetailOpac.aspx?id=151632
Napu, N., & Manipuspika, Y. S. (2025). Pengantar metodologi penelitian dalam kajian penerjemahan. Deepublish. https://deepublishstore.com/produk/buku-pengantar-metodologi-penelitian-dalam-kajian-penerjemahan/
Nitzke, J., & Hansen-Schirra, S. (2021). A short guide to post-editing. Language Science Press. https://doi.org/10.5281/zenodo.5646896
O’Neill, R., & Russell, A. (2019). Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly. Australasian Journal of Educational Technology, 35(1), 42-56. https://doi.org/10.14742/ajet.3795
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42, 533-544. https://doi.org/10.1007/s10488-013-0528-y
Ranalli, J., & Yamashita, T. (2022). Automated written corrective feedback: Error-correction performance and timing of delivery. Language Learning & Technology, 26(1), 1-25. https://doi.org/10.64152/10125/73465
Raunak, V., Sharaf, A., Wang, Y., Awadalla, H., & Menezes, A. (2023). Leveraging GPT-4 for automatic translation post-editing. In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 12009-12024). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-emnlp.804
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998-6008. https://doi.org/10.48550/arXiv.1706.03762
Wang, Y., & Daghigh, A. J. (2024). Effect of text type on translation effort in human translation and neural machine translation post-editing processes: Evidence from eye-tracking and keyboard-logging. Perspectives, 32(5), 961-976. https://doi.org/10.1080/0907676X.2023.2219850
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, Ł., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., … Dean, J. (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv. https://doi.org/10.48550/arXiv.1609.08144
Zhang, Z. V., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90-102. https://doi.org/10.1016/j.asw.2018.02.004
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Ismiyati Hasiru, Novriyanto Napu, Suleman S. Bouti
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
