Augmented reality integrated chatbot to improve learning outcomes in secondary school students

Reno Nurdiyanto; Henry Praherdhiono; Made Duananda Kartika Degeng

doi:10.17509/jik.v21i2.69108

Abstract

The major challenge in learning science is that it is abstract and complex for some students to understand. For example, elements and compounds cannot be seen directly in very micro-sized objects; therefore, visualizing this concept can be challenging. Another challenge is the limited time for teachers to answer questions, provide explanations, and provide feedback during the learning process. This study aims to develop augmented reality learning tools integrated with large language models in the format of a chatterbot text-to-text that can be utilized on the topic of elements and compounds. The development model that has been implemented is adapted from the Lee and Owen model, which includes need analysis, front-end analysis, design, development, implementation, and assessment. Media and content experts have assessed the technology's suitability for classroom learning for 8th-grade students of Salafiyah Junior High School in Pekalongan. Based on field implementation, this technology obtained a positive perspective and indicated an improvement in learning outcomes on element and compound topics. The integration of AR and ChatBot contributes significantly to the innovative classroom learning process and creates a learning experience that suits the needs of students in this digital era.

Abstrak

Kendala utama dalam pembelajaran IPA yaitu abstrak dan sulit untuk dipahami oleh beberapa siswa. Contohnya konsep mengenai unsur dan senyawa yang tidak bisa dilihat dalam bentuk secara langsung dalam objek yang berukuran sangat kecil, sehingga memvisualisasikan konsep ini bisa menjadi tantangan. Tantangan lain berupa waktu yang terbatas untuk guru dalam siswa dalam menjawab pertanyaan, memberikan penjelasan, dan memberikan umpan balik selama proses pembelajaran. Studi ini bertujuan untuk mengembangkan media pembelajaran berupa augmented reality terintegrasi large language models dalam bentuk chatterbot teks ke teks sehingga dapat digunakan pada topik tentang unsur dan senyawa. Model pengembangan yang telah dilakukan diadaptasi model Lee dan Owen, yang meliputi analisis kebutuhan, analisis awal akhir, desain, pengembangan, implementasi, dan penilaian. Ahli media dan ahli materi telah menyatakan kelayakan untuk digunakan dalam pembelajaran kelas pada siswa kelas 8 SMP Salafiyah Pekalongan. Berdasarkan implementasi lapangan, teknologi ini memperoleh perspektif yang positif dan menunjukkan adanya peningkatan hasil belajar pada materi unsur dan senyawa. Integrasi dari AR dan Chatbot memberikan kontribusi signifikan bagi sebagai media pembelajaran dalam proses pembelajaran kelas yang inovatif dan menciptakan pengalaman pembelajaran yang sesuai dengan kebutuhan siswa di era digital saat ini.

Kata Kunci: artificial intelligence; large language models; pembelajaran IPA; realitas berimbuh

Keywords:

artificial intelligence augmented reality large language model science learning

References

Al-Shloul, T., Mazhar, T., Abbas, Q., Iqbal, M., Ghadi, Y. Y., Shahzad, T., Mallek, F., & Hamam, H. (2024). Role of activity-based learning and ChatGPT on students’ performance in education. Computers and Education: Artificial Intelligence, 6(1), 10-29.

Alfianti, A., Kuswanto, H., Rahmat, A. D., & Nurdiyanto, R. (2023). Development of DICTY-AR integrated local wisdom to improve multiple representation and problem-solving skills. International Journal of Information and Education Technology, 13(9), 383-390.

Alizkan, U., Wibowo, F. C., Sanjaya, L., Kurniawan, B. R., & Prahani, B. K. (2021). Trends of augmented reality in science learning: A review of the literature. Journal of Physics: Conference Series, 19(1), 12-30.

Ardyansyah, A., & Rahayu, S. (2023). Development and implementation of augmented reality-based card game learning media with environmental literacy in improving students’ understanding of carbon compounds. Orbital: The Electronic Journal of Chemistry.15(2), 118-126

Bringula, R., Reguyal, J. J., Tan, D. D., & Ulfa, S. (2021). Mathematics self-concept and challenges of learners in an online learning environment during COVID-19 pandemic. Smart Learning Environments, 8(1), 22-36.

Brown, B., Boda, P., Ribay, K., Wilsey, M., & Perez, G. (2021). A technological bridge to equity: How VR designed through culturally relevant principles impact students’ appreciation of science. Learning, Media and Technology, 46(4), 564-584.

Cai, S., Jiao, X., Li, J., Jin, P., Zhou, H., & Wang, T. (2022). Conceptions of learning science among elementary school students in AR learning environment: A case study of “The Magic Sound.” Sustainability, 14(11), 67-83.

Chamorro-Atalaya, O., Olivares-Zegarra, S., Sobrino-Chunga, L., Guerrero-Carranza, R., Vargas-Diaz, A., Huarcaya-Godoy, M., Rasilla-Rovegno, J., Suarez-Bazalar, R., Poma-Garcia, J., & Cruz-Telada, Y. (2023). Application of the chatbot in University Education: A bibliometric analysis of indexed scientific production in SCOPUS, 2013-2023. International Journal of Learning, Teaching and Educational Research, 22(7), 281-304.

Chang, H.-Y., Liang, J.-C., & Tsai, C.-C. (2020). Students’ context-specific epistemic justifications, prior knowledge, engagement, and socioscientific reasoning in a mobile augmented reality learning environment. Journal of Science Education and Technology, 29(3), 399-408.

Chaturvedi, I., Cambria, E., & Welsch, R. E. (2023). Teaching simulations supported by artificial intelligence in the real world. Education Sciences, 13(2), 187-200.

Chen, T. C., Multala, E., Kearns, P., Delashaw, J., Dumont, A., Maraganore, D., & Wang, A. (2023). Assessment of ChatGPT’s performance on neurology written board examination questions. BMJ Neurology Open, 5(2), 530-543.

Chuang, C. H., Lo, J. H., & Wu, Y. K. (2023). Integrating chatbot and augmented reality technology into biology learning during COVID-19. Electronics, 12(1), 222-235.

Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444-452.

Eager, B., Brunton, R., & University of Tasmania, Australia. (2023). Prompting higher education towards AI-augmented teaching and learning practice. Journal of University Teaching and Learning Practice, 20(5), 25-36.

Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education, 9(1), 68-85.

Giray, L. (2023). Prompt engineering with ChatGPT: A guide for academic writers. Annals of Biomedical Engineering, 51(12), 2629-2633.

Hamid, A., Setyosari̇, P., Ulfa, S., & Kuswandi̇, D. (2019). The implementation of mobile seamless learning strategy in mastering students’ concepts for elementary school. Journal for the Education of Gifted Young Scientists, 7(4), 967-982.

Hidayati, N., Zubaidah, S., Suarsini, E., & Praherdhiono, H. (2020). Cognitive learning outcomes: Its relationship with communication skills and collaboration skills through digital mind maps-integrated PBL. International Journal of Information and Education Technology, 10(6), 443-448.

İbili, E., Çat, M., Resnyansky, D., Şahin, S., & Billinghurst, M. (2020). An assessment of geometry teaching supported with augmented reality teaching materials to enhance students’ 3D geometry thinking skills. International Journal of Mathematical Education in Science and Technology, 51(2), 224-246.

Karakus, M., Ersozlu, A., & Clark, A. C. (2019). Augmented reality research in education: A bibliometric study. Eurasia Journal of Mathematics, Science and Technology Education, 15(10). 29-39.

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., … & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 10(3), 10-22.

Korzynski, P., Mazurek, G., Krzypkowska, P., & Kurasinski, A. (2023). Artificial intelligence prompt engineering as a new digital competence: Analysis of generative AI technologies such as ChatGPT. Entrepreneurial Business and Economics Review, 11(3), 25-37.

Krismanto, W., Setyosari, P., Kuswandi, D., & Praherdhiono, H. (2022). Social media-based professional learning: What are teachers doing in it?. Qualitative Research in Education, 11(1), 89-116.

Lah, N. H. C., Senu, M. S. Z. M., Jumaat, N. F., Phon, D. N. E., Hashim, S., & Zulkifli, N. N. (2024). Mobile augmented reality in learning chemistry subject: An evaluation of science exploration. International Journal of Evaluation and Research in Education (IJERE), 13(2), 107-120.

Lambe, L., Degeng, I. N. S., Cahyono, B. Y., & Praherdiono, H. (2023). The effects of the application of graphic organizers on EFL students’ ability to write opinion essays and self-efficacy. Journal of Language Teaching and Research, 14(6), 502-514.

Liu, L. (2023). Analyzing the text contents produced by ChatGPT: Prompts, feature-components in responses, and a predictive model. Journal of Educational Technology Development and Exchange, 16(1), 49-70.

Lo, C. K. (2023). What Is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410-430.

Oktariani, O., Febliza, A., & Sari, Y. (2020). Konten materi kimia dalam standar isi pembelajaran IPA SMP berdasarkan K-13. Perspektif Pendidikan dan Keguruan, 11(2), 11-16.

Piccolo, S. R., Denny, P., Luxton-Reilly, A., Payne, S. H., & Ridge, P. G. (2023). Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. Plos Computational Biology, 19(9), 101-111.

Ramansyah, W., Praherdhiono, H., Sudana D, I. N., & Kuswandi, D. (2023). A gamified MOOC: The development of an interactive moodle-based learning environment. Ubiquitous Learning: An International Journal, 17(1), 93-119.

Rodríguez, F. C., Krapp, L. F., Dal Peraro, M., & Abriata, L. A. (2022). Visualization, interactive handling and simulation of molecules in commodity augmented reality in web browsers using moleculARweb’s virtual modeling kits. Chimia, 76(1), 145-150.

Rudolph, J., Tan, S., & Tan, S. (2023). War of the chatbots Bard, Bing Chat, ChatGPT, Ernie and beyond: The new AI gold rush and its impact on higher education. Journal of Applied Learning & Teaching, 6(1), 37-50.

Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare, 11(6), 887-900.

Scherr, R., Halaseh, F. F., Spina, A., Andalib, S., & Rivera, R. (2023). ChatGPT interactive medical simulations for early clinical education: Case study. JMIR Medical Education, 9(1), 98-107.

Suartama, I. K., Setyosari, P., Sulthoni, S., & Ulfa, S. (2019). Development of an Instructional design model for mobile blended learning in higher education. International Journal of Emerging Technologies in Learning (iJET), 14(16), 4-20.

Suartama, I. K., Setyosari, P., Sulthoni, S., & Ulfa, S. (2020). Development of ubiquitous learning environment based on moodle learning management system. International Journal of Interactive Mobile Technologies (iJIM), 14(14), 182-200.

Subiyantoro, S., Degeng, I. N. S., Kuswandi, D., & Ulfa, S. (2024). Developing gamified learning management systems to increase student engagement in online learning environments. International Journal of Information and Education Technology, 14(1), 26-33.

Sulistiani, I. R., Setyosari, P., Sa’dijah, C., & Praherdhiono, H. (2023). Technology integration through acceptance of e-learning among preservice teachers. Indonesian Journal of Electrical Engineering and Computer Science, 31(3), 18-30.

Thaariq, Z. Z. A., Nurdiyanto, R., Karima, U., Putri, C. E., Utomo, D. A. D. W., Kesuma, D. W., & Kuswandi, D. (2023). Conducting hybrid training for teacher professionalism at SMP Wahid Hasyim Malang. Journal of Community Practice and Social Welfare, 3(2), 1-11.

Thaariq, Z. Z. A., Nurdiyanto, R., & Sulfa, D. M. (2023). Masalah-masalah dalam paradigma pembelajaran sains modern. Jurnal Nyanadassana: Jurnal Penelitian Pendidikan, Sosial dan Keagamaan, 2(2), 112-121.

Thaariq, Z. Z. A., Yulianto, M. F., & Nurdiyanto, R. (2023). Construction of an Adaptive Blended Curriculum (ABC) model in implementing local content curriculum. Inovasi Kurikulum, 20(2), 177-192.

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15-28.

Turan, Z., & Atila, G. (2021). Augmented reality technology in science education for students with specific learning difficulties: Its effect on students’ learning and views. Research in Science & Technological Education, 39(4), 506-524.

Wang, J. T. H. (2023). Is the laboratory report dead? AI and ChatGPT. Microbiology Australia, 44(3), 144-148.

Wörner, S., Kuhn, J., & Scheiter, K. (2022). The best of two worlds: A systematic review on combining real and virtual experiments in science education. Review of Educational Research, 92(6), 911-952.

Wu, T. T., & Wu, Y. T. (2020). Applying project-based learning and SCAMPER teaching strategies in engineering education to explore the influence of creativity on cognition, personal motivation, and personality traits. Thinking Skills and Creativity, 35(1), 16-31.

Yang, F. Y., & Wang, H. Y. (2023). Tracking visual attention during learning of complex science concepts with augmented 3D visualizations. Computers & Education, 19(3), 46-59.

Zubaidah, S., Suarsini, E., & Praherdhiono, H. (2020). Cognitive learning outcomes: Its relationship with communication skills and collaboration skills through digital mind maps-integrated PBL. International Journal of Information and Education Technology, 10(6), 443-448.