A Polya-Aligned Prompting Protocol for ChatGPT Scaffolding: Evidence from Eighth-Grade Systems-of-Linear-Equations Problem Solving
Article Sidebar
Main Article Content
Abstract
Generative artificial intelligence offers new opportunities to scaffold students’ mathematical reasoning, yet rigorous evidence of its impact on secondary students’ problem-solving remains limited. This study examined whether ChatGPT-driven adaptive learning improves eighth-grade students’ problem-solving performance on systems of linear equations in two variables (SPLDV) compared with conventional instruction. A pre–post-test control-group design was implemented with 47 eighth-grade students in Parepare, Indonesia (experimental n = 24; control n = 23) during the 2024/2025 academic year. The experimental group used ChatGPT as an adaptive tutor aligned with Polya’s stages (understand, plan, execute, look back) through guided prompts, hints, and feedback. In contrast, the control group received a lecture and practice. Students completed a six-item contextual SPLDV test scored with a Polya-based rubric. Between-group differences were tested on post-test scores and normalized gains after verifying normality and homogeneity assumptions. The experimental group achieved higher post-test performance (M = 68.33) than the control group (M = 59.57), with a significant difference (p = 0.019; η² = 0.117). Learning gains were also larger in the experimental group (mean N-gain = 0.34, medium) than in the control group (0.21, low; p = 0.001; η² = 0.372). Indicator-level patterns suggested the greatest improvements in understanding the problem and carrying out the plan, whereas devising a plan remained the most challenging stage in both groups. These findings indicate that ChatGPT-based adaptive scaffolding can enhance students’ mathematical problem-solving on SPLDV, but explicit teacher-guided routines are needed to strengthen strategic planning and the critical evaluation of AI outputs.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Copyright of the published article belongs to the authors and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 (CC BY SA) International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access).
References
[1] M. Jurdak, “Learning and Teaching Real World Problem Solving in School Mathematics,” Learn. Teach. Real World Probl. Solving Sch. Math., 2016, https://doi.org/10.1007/978-3-319-08204-2
[2] J. Nilimaa, “New Examination Approach for Real-World Creativity and Problem-Solving Skills in Mathematics,” Trends High. Educ., vol. 2, no. 3, pp. 477–495, 2023, https://doi.org/10.3390/higheredu2030028
[3] B. Trilling and C. Fadel, 21St Century Skills: Learning for Life in Our Times, vol. 47, no. 10. John Wiley & Sons, 2010. https://doi.org/10.5860/choice.47-5788
[4] OECD, “An OECD Learning Framework 2030,” The Future of Education and Labor. Springer, pp. 23–35, 2019. https://doi.org/10.1007/978-3-030-26068-2_3
[5] L. W. Anderson, D. R. Krathwohl, and B. S. Bloom, A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives. Addison Wesley Longman, Inc., 2001.
[6] S. M. Brookhart, How to assess higher-order thinking skills in your classroom. Ascd, 2010.
[7] J. Cai, “What Research Tells Us About Teaching Mathematics Through Problem Solving,” IEEE Sens. J., vol. 5, no. 2, pp. 273–276, 2005.
[8] E. G. Harskamp and C. J. M. Suhre, “Improving mathematical problem solving: A computerized approach,” Comput. Human Behav., vol. 22, no. 5, pp. 801–815, 2006, https://doi.org/10.1016/j.chb.2004.03.023
[9] H. J. Pol, E. G. Harskamp, C. J. M. Suhre, and M. J. Goedhart, “How indirect supportive digital help during and after solving physics problems can improve problem-solving abilities,” Comput. Educ., vol. 53, no. 1, pp. 34–50, 2009, https://doi.org/10.1016/j.compedu.2008.12.015
[10] F. K. Lester, “Thoughts about research on mathematical problem- solving instruction,” Math. Enthus., vol. 10, no. 1–2, pp. 245–278, 2013, https://doi.org/10.54870/1551-3440.1267
[11] K. Stacey, “The place of problem solving in contemporary mathematics curriculum documents,” J. Math. Behav., vol. 24, no. 3–4, pp. 341–350, 2005, https://doi.org/10.1016/j.jmathb.2005.09.004
[12] G. Polya, How to Solve It: A New Aspect of Mathematical Method. Princeton University Press, 2004.
[13] D. Jonassen, “Supporting Problem Solving in PBL,” Interdiscip. J. Probl. Learn., vol. 5, no. 2, pp. 95–119, 2011, https://doi.org/10.7771/1541-5015.1256
[14] K. Manasa Lakshmi, K. Michael Angelo, S. Naga Lakshmi, and P. Ramu, “Smart approach for e-health monitoring system,” in AIP Conference Proceedings, 2025, vol. 3162, no. 1, p. 20086. https://doi.org/10.1063/5.0249448
[15] I. Campo-Ruiz, “Artificial intelligence may affect diversity: architecture and cultural context reflected through ChatGPT, Midjourney, and Google Maps,” Humanit. Soc. Sci. Commun., vol. 12, no. 1, pp. 1–13, 2025, https://doi.org/10.1057/s41599-024-03968-5
[16] Q. Wang, T. Sun, and R. Li, “Does Artificial Intelligence (AI) enhance green economy efficiency? The role of green finance, trade openness, and R&D investment,” Humanit. Soc. Sci. Commun., vol. 12, no. 1, pp. 1–22, 2025, https://doi.org/10.1057/s41599-024-04319-0
[17] I. H. Cho and A. Chapagain, “Self-evolving artificial intelligence framework to better decipher short-term large earthquakes,” Sci. Rep., vol. 14, no. 1, p. 21934, 2024, https://doi.org/10.1038/s41598-024-72667-7
[18] G. Zhang, A. Raina, J. Cagan, and C. McComb, “A cautionary tale about the impact of AI on human design teams,” Des. Stud., vol. 72, p. 100990, 2021.
[19] S. Morandini, F. Fraboni, M. De Angelis, G. Puzzo, D. Giusino, and L. Pietrantoni, “the Impact of Artificial Intelligence on Workers’ Skills: Upskilling and Reskilling in Organisations,” Informing Sci., vol. 26, pp. 39–68, 2023, https://doi.org/10.28945/5078
[20] H. Ackermann et al., “Physical embodiment and anthropomorphism of AI tutors and their role in student enjoyment and performance,” npj Sci. Learn., vol. 10, no. 1, p. 1, 2025, https://doi.org/10.1038/s41539-024-00293-z
[21] N. Bienefeld, E. Keller, and G. Grote, “AI Interventions to Alleviate Healthcare Shortages and Enhance Work Conditions in Critical Care: Qualitative Analysis,” J. Med. Internet Res., vol. 27, p. e50852, 2025, https://doi.org/10.2196/50852
[22] I. Roll and R. Wylie, “Evolution and Revolution in Artificial Intelligence in Education,” Int. J. Artif. Intell. Educ., vol. 26, no. 2, pp. 582–599, 2016, https://doi.org/10.1007/s40593-016-0110-3
[23] W. Holmes and I. Tuomi, “State of the art and practice in AI in education,” Eur. J. Educ., vol. 57, no. 4, pp. 542–570, 2022, https://doi.org/10.1111/ejed.12533
[24] T. Shahzad, T. Mazhar, M. U. Tariq, W. Ahmad, K. Ouahada, and H. Hamam, “A comprehensive review of large language models: issues and solutions in learning environments,” Discov. Sustain., vol. 6, no. 1, p. 27, 2025, https://doi.org/10.1007/s43621-025-00815-8
[25] T. Brown et al., “Language models are few-shot learners,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 1877–1901, 2020.
[26] N. Noster, S. Gerber, and H.-S. Siller, “Pre-Service Teachers’ Approaches in Solving Mathematics Tasks with ChatGPT,” Digit. Exp. Math. Educ., vol. 10, no. 3, pp. 543–567, 2024, https://doi.org/10.1007/s40751-024-00155-8
[27] E. Kasneci et al., “ChatGPT for good? On opportunities and challenges of large language models for education,” Learn. Individ. Differ., vol. 103, p. 102274, 2023, https://doi.org/10.1016/j.lindif.2023.102274
[28] L. Busuttil and J. Calleja, “Teachers’ Beliefs and Practices About the Potential of ChatGPT in Teaching Mathematics in Secondary Schools,” Digit. Exp. Math. Educ., vol. 11, no. 1, pp. 140–166, 2025, https://doi.org/10.1007/s40751-024-00168-3
[29] B. Pepin, N. Buchholtz, and U. Salinas-Hernández, “A Scoping Survey of ChatGPT in Mathematics Education,” Digit. Exp. Math. Educ., vol. 11, no. 1, pp. 9–41, 2025, https://doi.org/10.1007/s40751-025-00172-1
[30] K. Łodzikowski, P. W. Foltz, and J. T. Behrens, “Generative AI and Its Educational Implications,” in Postdigital Science and Education (Netherlands), vol. Part F3835, Springer, 2024, pp. 35–57. https://doi.org/10.1007/978-3-031-64487-0_2
[31] R. Zhou et al., “Exploring ChatGPT-Facilitated Scaffolding in Undergraduates’ Mathematical Problem Solving,” J. Comput. Assist. Learn., vol. 41, no. 4, p. e70077, 2025, https://doi.org/10.1111/jcal.70077
[32] B. A. Becker et al., “Generative AI in Introductory Programming,” Comput. Sci. Curricula 2023, Curric. Pract. Vol., vol. 1, p. 25, 2023.
[33] C. Fadel, W. Holmes, and M. Bialik, Artificial intelligence in education: Promises and implications for teaching and learning. The Center for Curriculum Redesign, Boston, MA, vol. 14, no. 4. Center for Curriculum Redesign, 2019.
[34] O. Zawacki-Richter, V. I. Marín, M. Bond, and F. Gouverneur, “Systematic review of research on artificial intelligence applications in higher education – where are the educators?,” Int. J. Educ. Technol. High. Educ., vol. 16, no. 1, pp. 1–27, 2019, https://doi.org/10.1186/s41239-019-0171-0
[35] D. Loewenberg Ball, M. H. Thames, and G. Phelps, “Content knowledge for teaching: What makes it special?,” J. Teach. Educ., vol. 59, no. 5, pp. 389–407, 2008, https://doi.org/10.1177/0022487108324554
[36] J. van de Pol, M. Volman, and J. Beishuizen, “Scaffolding in teacher-student interaction: A decade of research,” Educ. Psychol. Rev., vol. 22, no. 3, pp. 271–296, 2010, https://doi.org/10.1007/s10648-010-9127-6
[37] D. Holton and D. Clarke, “Scaffolding and metacognition,” Int. J. Math. Educ. Sci. Technol., vol. 37, no. 2, pp. 127–143, 2006, https://doi.org/10.1080/00207390500285818
[38] E. A. Davis and N. Miyake, “Explorations of Scaffolding in Complex Classroom Systems,” in Scaffolding: A special issue of the journal of the learning sciences, Psychology Press, 2018, pp. 265–272. https://doi.org/10.4324/9780203764411-1
[39] J. Hattie, M. Gan, and C. Brooks, “Instruction Based on Feedback,” in Handbook of Research on Learning and Instruction, Second edition, Routledge, 2016, pp. 290–324. https://doi.org/10.4324/9781315736419-23
[40] V. J. Shute, “Focus on Formative Feedback,” ETS Res. Rep. Ser., vol. 2007, no. 1, pp. i–47, 2007, https://doi.org/10.1002/j.2333-8504.2007.tb02053.x
[41] B. Wisniewski, K. Zierer, and J. Hattie, “The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research,” Front. Psychol., vol. 10, p. 487662, 2020, https://doi.org/10.3389/fpsyg.2019.03087
[42] F. M. Van der Kleij, R. C. W. Feskens, and T. J. H. M. Eggen, “Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes: A Meta-Analysis,” Rev. Educ. Res., vol. 85, no. 4, pp. 475–511, 2015, https://doi.org/10.3102/0034654314564881
[43] Z. A. Pardos and S. Bhandari, “ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills,” PLoS One, vol. 19, no. 5 May, p. e0304013, 2024, https://doi.org/10.1371/journal.pone.0304013
[44] A. Gupta, J. Reddig, T. Calò, D. Weitekamp, and C. J. MacLellan, “Beyond Final Answers: Evaluating Large Language Models for Math Tutoring,” in Lecture Notes in Computer Science, 2025, vol. 15877 LNAI, pp. 323–337. https://doi.org/10.1007/978-3-031-98414-3_23
[45] M. Turmuzi, S. Azmi, and N. M. I. Kertiyani, “ChatGPT in school mathematics education: A systematic review of opportunities, challenges, and pedagogical implications,” Teach. Teach. Educ., vol. 170, p. 105286, 2026, https://doi.org/10.1016/j.tate.2025.105286
[46] T. Jiang, S. Huang, Z. Luan, D. Wang, and F. Zhuang, “Scaling Sentence Embeddings with Large Language Models,” in EMNLP 2024 - 2024 Conference on Empirical Methods in Natural Language Processing, Findings of EMNLP 2024, 2024, pp. 3182–3196. https://doi.org/10.18653/v1/2024.findings-emnlp.181
[47] G. Polya, How to solve it: A new aspect of mathematical method. Princeton university press, 2014. https://doi.org/10.2307/2306109
[48] G. Polya, How to Solve It. Princeton Press, 2019. https://doi.org/10.2307/j.ctvc773pk
[49] F. Fitriani, R. Hayati, S. Sugeng, S. Srimuliati, and T. Herman, “Students’ Ability To Solve Mathematical Problems Through Polya Steps,” J. Eng. Sci. Technol., vol. 17, pp. 25–32, 2022.
[50] Z. Aba-Oli, K. Koyas, and A. Husen, “Higher-order thinking skills-oriented problem-based learning interventions in mathematics: A systematic literature review,” Sch. Sci. Math., vol. 125, no. 3, pp. 214–231, 2025, https://doi.org/10.1111/ssm.12676
[51] M. Sadak, L. Incikabi, F. Ulusoy, and M. Pektas, “Investigating mathematical creativity through the connection between creative abilities in problem posing and problem solving,” Think. Ski. Creat., vol. 45, p. 101108, 2022, https://doi.org/10.1016/j.tsc.2022.101108
[52] L. Menezes, H. Oliveira, and A. P. Canavarro, “Inquiry-based mathematics teaching: The case of Célia,” in Educational Paths to Mathematics: A CIEAEM Sourcebook, Springer, 2015, pp. 305–321.
[53] J. F. Hair, W. C. Black, B. J. Babin, R. E. Anderson, and R. L. Tatham, Multivariate Data Analysis, 8th ed. Edinburgh Gate, Harlow: Pearson Education Limited, 2019.
[54] B. Tabachnick and L. Fidell, Using Multivariate Statistics Title: Using multivariate statistics, vol. 5, no. 7th. Boston, Massachusetts, United State: Allyn & Bacon/Pearson Education, 2019.
[55] K. S. Taber, “The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education,” Res. Sci. Educ., vol. 48, no. 6, pp. 1273–1296, 2018, https://doi.org/10.1007/s11165-016-9602-2
[56] R. R. Hake, “Analyzing change/gain scores.” 1999.
[57] F. Fui-Hoon Nah, R. Zheng, J. Cai, K. Siau, and L. Chen, “Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration,” Journal of information technology case and application research, vol. 25, no. 3. Taylor & Francis, pp. 277–304, 2023. https://doi.org/10.1080/15228053.2023.2233814
[58] C. Zhai, S. Wibowo, and L. D. Li, “The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: a systematic review,” Smart Learn. Environ., vol. 11, no. 1, p. 28, 2024, https://doi.org/10.1186/s40561-024-00316-7
[59] M. Doorman, P. Drijvers, T. Dekker, M. van den Heuvel-Panhuizen, J. de Lange, and M. Wijers, “Problem solving as a challenge for mathematics education in The Netherlands,” ZDM Math. Educ., vol. 39, no. 5, pp. 405–418, 2007, https://doi.org/10.1007/s11858-007-0043-2
[60] K. Reiss and G. Törner, “Problem solving in the mathematics classroom: The German perspective,” ZDM - Int. J. Math. Educ., vol. 39, no. 5–6, pp. 431–441, 2007, https://doi.org/10.1007/s11858-007-0040-5
[61] C. Mah, H. Walker, L. Phalen, S. Levine, S. W. Beck, and J. Pittman, “Beyond CheatBots: Examining Tensions in Teachers’ and Students’ Perceptions of Cheating and Learning with ChatGPT,” Educ. Sci., vol. 14, no. 5, p. 500, 2024, https://doi.org/10.3390/educsci14050500
[62] M. Kapur, “Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning,” Educ. Psychol., vol. 51, no. 2, pp. 289–299, 2016, https://doi.org/10.1080/00461520.2016.1155457
[63] H. Jo, “From concerns to benefits: a comprehensive study of ChatGPT usage in education,” Int. J. Educ. Technol. High. Educ., vol. 21, no. 1, p. 35, 2024, https://doi.org/10.1186/s41239-024-00471-4
[64] G.-J. Hwang and Y.-F. Tu, “Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review,” Mathematics, vol. 9, no. 6, p. 584, 2021, https://doi.org/10.3390/su142013185
[65] O. Taani and S. Alabidi, “ChatGPT in education: benefits and challenges of ChatGPT for mathematics and science teaching practices,” Int. J. Math. Educ. Sci. Technol., pp. 1–30, 2024, https://doi.org/10.1080/0020739X.2024.2357341
[66] S. A. Costa, “Reasoning, Argumentation, and Mathematical Learning: How ChatGPT and AI Can Knowledge Build,” in Mathematics and Education in an AI Era: Cognitive Science, Technological, and Semiotic Perspectives, Springer, 2025, pp. 89–115. https://doi.org/10.1007/978-3-031-80696-4_6
[67] S. Kanbul, I. Adamu, and Y. B. Mohammed, “A Global Outlook on AI-Predicted Impacts of ChatGPT on Contemporary Education,” SAGE Open, vol. 14, no. 3, p. 21582440241266370, 2024, https://doi.org/10.1177/21582440241266370
[68] Z. K. Szabo, P. Körtesi, J. Guncaga, D. Szabo, and R. Neag, “Examples of problem-solving strategies in mathematics education supporting the sustainability of 21st-century skills,” Sustain., vol. 12, no. 23, pp. 1–28, 2020, https://doi.org/10.3390/su122310113
[69] A. Jančařík, J. Michal, and J. Novotná, “Using AI Chatbot for Math Tutoring,” J. Educ. Cult. Soc., vol. 14, no. 2, pp. 285–296, 2023, https://doi.org/10.15503/jecs2023.2.285.296
[70] D. Lee and S. Yeo, “Developing an AI-based chatbot for practicing responsive teaching in mathematics,” Comput. Educ., vol. 191, p. 104646, 2022, https://doi.org/10.1016/j.compedu.2022.104646
[71] I. Sasson, I. Yehuda, and N. Malkinson, “Fostering the skills of critical thinking and question-posing in a project-based learning environment,” Think. Ski. Creat., vol. 29, pp. 203–212, 2018, https://doi.org/10.1016/j.tsc.2018.08.001
[72] P. Kwangmuang, S. Jarutkamolpong, W. Sangboonraung, and S. Daungtod, “The development of learning innovation to enhance higher order thinking skills for students in Thailand junior high schools,” Heliyon, vol. 7, no. 6, 2021, https://doi.org/10.1016/j.heliyon.2021.e07309
[73] M. V. Pop, G. Tonț, F.-V. Flonta, and M. Flore, “Agentic AI in STEM Education: Enhancing Cognitive Flexibility and Workforce Readiness,” BRAIN. Broad Res. Artif. Intell. Neurosci., vol. 16, no. 1 Sup1, p. 239, 2025, https://doi.org/10.70594/brain/16.s1/20
[74] L. Y. Tan, S. Hu, D. J. Yeo, and K. H. Cheong, “Artificial intelligence-enabled adaptive learning platforms: A review,” Comput. Educ. Artif. Intell., vol. 9, p. 100429, 2025, https://doi.org/10.1016/j.caeai.2025.100429