Automated Feedback Through Language Models and Written Production in Primary Education Students from Guaranda, Ecuador
DOI:
https://doi.org/10.70577/cieninter.v4i3.43Keywords:
automated feedback, language models, written production, primary education, artificial intelligence in education, formative assessment.Abstract
The integration of large language models in education has sparked growing debate regarding their potential to support writing processes at the earliest levels of schooling. This study examined the effects of automated feedback generated by a language model on the quality of written production among fifth- and sixth-grade students in public schools in Guaranda, Bolívar Province, Ecuador. A quasi-experimental design was used with control and experimental groups, pretest and posttest measurements, and a sample of 127 students distributed across four classrooms from two schools. The experimental group received feedback generated by an adapted language model, while the control group continued with teacher-only feedback. Written production was assessed using a validated analytical rubric covering five dimensions: coherence, cohesion, lexical adequacy, grammatical accuracy, and textual structure. Results showed statistically significant differences favoring the experimental group in coherence (p = .003), cohesion (p = .011), and grammatical accuracy (p = .007), with medium effect sizes according to Cohen's d. No differences were found in lexical adequacy or textual structure. Semi-structured interviews with 12 teachers revealed mixed perceptions: they valued the immediacy of the feedback but expressed concerns about technological dependence and the model's limited contextual sensitivity. Findings suggest that automated feedback can complement, though not replace, the teacher's role in writing instruction at the primary level.
Downloads
References
Baker, R. S. y Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32(4), 1052–1092. https://doi.org/10.1007/s40593-021-00285-9 DOI: https://doi.org/10.1007/s40593-021-00285-9
Belland, B. R. (2014). Scaffolding: Definition, current debates, and future directions. En J. M. Spector, M. D. Merrill, J. Elen y M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4.ª ed., pp. 505–518). Springer. https://doi.org/10.1007/978-1-4614-3185-5_39 DOI: https://doi.org/10.1007/978-1-4614-3185-5_39
Braun, V. y Clarke, V. (2021). Thematic analysis: A practical guide. SAGE Publications. DOI: https://doi.org/10.4135/9781036232078.n9
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2.ª ed.). Lawrence Erlbaum Associates.
Crompton, H. y Burke, D. (2023). Artificial intelligence in higher education: The state of the field. International Journal of Educational Technology in Higher Education, 20(1), 22. https://doi.org/10.1186/s41239-023-00392-8 DOI: https://doi.org/10.1186/s41239-023-00392-8
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y.-S., Gašević, D. y Chen, G. (2023). Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv. https://doi.org/10.48550/arXiv.2310.01783 DOI: https://doi.org/10.1109/ICALT58122.2023.00100
Escalante, J., Hernández, R. y Valdivia, M. (2023). Retroalimentación automatizada mediante ChatGPT en textos argumentativos de educación secundaria. Revista Iberoamericana de Educación, 92(1), 45–63. https://doi.org/10.35362/rie9215741
Flotts, M. P., Manzi, J., Polloni, M. P., Carrasco, M., Zambra, C. y Abarzúa, A. (2021). Aportes para la enseñanza de la escritura a partir de resultados de evaluaciones estandarizadas en América Latina. Revista Latinoamericana de Estudios Educativos, 51(2), 169–198.
Graham, S. (2022). A revised writer(s)-within-community model of writing. Educational Psychologist, 57(2), 85–110. https://doi.org/10.1080/00461520.2021.2001387
Graham, S., Hebert, M. y Harris, K. R. (2022). Formative assessment and writing: A meta-analysis. The Elementary School Journal, 122(4), 525–550. https://doi.org/10.1086/719856 DOI: https://doi.org/10.1086/719856
Hattie, J. y Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487 DOI: https://doi.org/10.3102/003465430298487
Holmes, W., Bialik, M. y Fadel, C. (2022). Artificial intelligence in education: Promises and implications for teaching and learning (2.ª ed.). Center for Curriculum Redesign.
Instituto Nacional de Estadística y Censos [INEC]. (2022). Proyecciones poblacionales a nivel cantonal 2020-2025. INEC.
Instituto Nacional de Evaluación Educativa [INEVAL]. (2023). Resultados nacionales Ser Estudiante 2023: Educación General Básica. INEVAL.
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Kruber, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., ... Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274 DOI: https://doi.org/10.1016/j.lindif.2023.102274
Ministerio de Educación del Ecuador. (2021). Currículo priorizado con énfasis en competencias comunicacionales, matemáticas, digitales y socioemocionales: Subnivel Medio de Educación General Básica. Ministerio de Educación.
Nazaretsky, T., Ariely, M., Cukurova, M. y Alexandron, G. (2022). Teachers' trust in AI-powered educational technology and a professional development program to improve it. British Journal of Educational Technology, 53(4), 914–931. https://doi.org/10.1111/bjet.13232 DOI: https://doi.org/10.1111/bjet.13232
OpenAI. (2023). GPT-4 technical report. arXiv. https://doi.org/10.48550/arXiv.2303.08774
OpenAI. (2024). GPT-4o system card. OpenAI. https://openai.com/index/gpt-4o-system-card/
Organisation for Economic Co-operation and Development [OECD]. (2021). OECD digital education outlook 2021: Pushing the frontiers with artificial intelligence, blockchain and robots. OECD Publishing. https://doi.org/10.1787/589b283f-en DOI: https://doi.org/10.1787/589b283f-en
Peterson, S. S. y McClay, J. K. (2021). Teachers' written comments and student writing in elementary classrooms. Written Communication, 38(3), 391–428. https://doi.org/10.1177/07410883211006360
Sánchez-Prieto, J. C., Trujillo-Torres, J. M., Gómez-García, M. y Gómez-García, G. (2022). Mapping research on artificial intelligence in education in Latin America: A bibliometric analysis. Education Sciences, 12(12), 918. https://doi.org/10.3390/educsci12120918 DOI: https://doi.org/10.3390/educsci12120918
Shadish, W. R., Cook, T. D. y Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
Sotomayor, C., Gómez, G., Jéldrez, E. y Bedwell, P. (2021). Análisis de la retroalimentación escrita de profesores a textos narrativos de estudiantes de educación primaria. Estudios Pedagógicos, 47(1), 281–299. https://doi.org/10.4067/S0718-07052021000100281 DOI: https://doi.org/10.4067/S0718-07052021000100281
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W. y Warschauer, M. (2024). Comparing the quality of human and ChatGPT feedback on students' writing. Learning and Instruction, 91, 101894. https://doi.org/10.1016/j.learninstruc.2024.101894 DOI: https://doi.org/10.1016/j.learninstruc.2024.101894
UNESCO. (2021). AI and education: Guidance for policy-makers. UNESCO Publishing. https://doi.org/10.54675/PCSP7350 DOI: https://doi.org/10.54675/PCSP7350
UNESCO. (2023). Guidance for generative AI in education and research. UNESCO Publishing. https://doi.org/10.54675/EWZQ9977
Valverde-Berrocoso, J., del Carmen Garrido-Arroyo, M., Burgos-Videla, C. G. y Morales-Cevallos, M. B. (2022). Trends in educational research about e-learning: A systematic literature review (2009–2018). Sustainability, 12(12), 5153. https://doi.org/10.3390/su12125153 DOI: https://doi.org/10.3390/su12125153
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. y Polosukhin, I. (2017). Attention is all you need. En I. Guyon et al. (Eds.), Advances in Neural Information Processing Systems 30 (pp. 5998–6008). Curran Associates.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Warschauer, M. y Grimes, D. (2008). Automated writing assessment in the classroom. Pedagogies: An International Journal, 3(1), 22–36. https://doi.org/10.1080/15544800701771580 DOI: https://doi.org/10.1080/15544800701771580
Wisniewski, B., Zierer, K. y Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087. https://doi.org/10.3389/fpsyg.2019.03087 DOI: https://doi.org/10.3389/fpsyg.2019.03087
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y. y Gašević, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90–112. https://doi.org/10.1111/bjet.13370 DOI: https://doi.org/10.1111/bjet.13370
Zheng, L., Niu, J., Zhong, L. y Gyasi, J. F. (2023). The effectiveness of artificial intelligence on learning outcomes and learning persistence in primary and secondary education: A meta-analysis. Computers & Education, 203, 104893. https://doi.org/10.1016/j.compedu.2023.104893
Zawacki-Richter, O., Marín, V. I., Bond, M. y Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education: Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0 DOI: https://doi.org/10.1186/s41239-019-0171-0
Baidoo-Anu, D. y Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62. https://doi.org/10.61969/jai.1337500 DOI: https://doi.org/10.61969/jai.1337500
Mollick, E. R. y Mollick, L. (2023). Using AI to implement effective teaching strategies in classrooms: Five strategies, including prompts. The Wharton School Research Paper. https://doi.org/10.2139/ssrn.4391243 DOI: https://doi.org/10.2139/ssrn.4391243
Celik, I. (2023). Towards intelligent-TPACK: An empirical study on teachers' professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138, 107468. https://doi.org/10.1016/j.chb.2022.107468 DOI: https://doi.org/10.1016/j.chb.2022.107468
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J. y Li, Y. (2021). A review of artificial intelligence (AI) in education from 2010 to 2020. Complexity, 2021, 8812542. https://doi.org/10.1155/2021/8812542
Su, J., Zhong, Y. y Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence, 3, 100065. https://doi.org/10.1016/j.caeai.2022.100065 DOI: https://doi.org/10.1016/j.caeai.2022.100065
Kim, N. J. y Kim, M. K. (2022). Teacher's perceptions of using an artificial intelligence-based educational tool for scientific writing. Frontiers in Education, 7, 755914. https://doi.org/10.3389/feduc.2022.755914 DOI: https://doi.org/10.3389/feduc.2022.755914
Luckin, R. y Cukurova, M. (2019). Designing educational technologies in the age of AI: A learning sciences-driven approach. British Journal of Educational Technology, 50(6), 2824–2838. https://doi.org/10.1111/bjet.12861 DOI: https://doi.org/10.1111/bjet.12861
Mao, J., Chen, B. y Liu, J. C. (2024). Generative AI and its impact on personalized intelligent tutoring systems. Education and Information Technologies, 29(1), 1443–1464. https://doi.org/10.1007/s10639-024-12580-w
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Hortencia Magdalena Bonilla Verdezoto, Zoila América García García, Lidia Ivonne Basantes Silva , Ximena Graciela Carguachi Mancheno
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
