Pengembangan Game Tower Defense Berbasis Phaser Engine dengan Implementasi Q-Learning untuk Musuh Adaptif
DOI:
https://doi.org/10.62712/juktisi.v5i2.1494Keywords:
reinforcement learning, Q-learning, adaptive enemy, tower defense, phaser engine, game development life cycle, artificial intelligenceAbstract
Kualitas perilaku musuh menjadi faktor penentu keterlibatan pemain pada game bergenre tower defense, namun algoritma pathfinding klasik seperti A* tidak mempertimbangkan tingkat ancaman di sepanjang rute, sehingga musuh selalu menempuh rute yang sama meskipun terdapat pertahanan sangat kuat. Penelitian ini bertujuan mengimplementasikan algoritma Q-Learning pada unit musuh dalam game tower defense berbasis phaser engine untuk menghasilkan agen yang adaptif terhadap perubahan ancaman. Agen dirancang dengan state berupa posisi grid dan level ancaman diskrit pada empat arah, ruang aksi empat arah pergerakan, serta fungsi reward berjenjang. Training dilakukan selama 5.000 episode dengan skema epsilon-greedy, dan kemampuan adaptasi diuji pada lima skenario konfigurasi menara, dibandingkan dengan agen non-adaptif (random walk dan A* statis). Hasil menunjukkan agen mencapai pembelajaran konvergen, dengan reward meningkat 96% dan success rate 99,84% selama training. Agen terbukti beradaptasi secara proporsional terhadap konfigurasi ancaman, ditunjukkan melalui perubahan pola jalur yang signifikan serta reaksi real-time terhadap menara yang muncul mendadak. Pada evaluasi akhir, agen mencapai success rate 100% dengan efisiensi jalur 1,15–1,47 kali jarak optimal. Dibandingkan random walk, efisiensi jalur agen 20–30 kali lebih baik, sedangkan terhadap A* statis terdapat trade-off wajar (selisih 15–47% langkah) demi mengurangi eksposur ancaman. Penelitian ini membuktikan bahwa Q-Learning tabular efektif menghasilkan perilaku musuh adaptif dengan komputasi ringan untuk platform browser.
Downloads
References
[1] S. B. Alaoui and A. Saoud, "How to discretize continuous state-action spaces in Q-learning: A symbolic control approach," in 2024 IEEE 63rd Conference on Decision and Control (CDC), 2024, pp. 8314–8319, doi: https://doi.org/10.1109/cdc56724.2024.10886490.
[2] J. Bergdahl, A. Sestini, and L. Gisslén, "Reinforcement learning for high-level strategic control in tower defense games," in 2024 IEEE Conference on Games (CoG), 2024, pp. 1–8, doi: https://doi.org/10.1109/cog60054.2024.10645621.
[3] A. Dias, J. Foleiss, and R. P. Lopes, "Reinforcement learning in tower defense," in Communications in Computer and Information Science, 2022, doi: https://doi.org/10.1007/978-3-030-95305-8_10.
[4] F. Ebrahimabadi, H. Kebriaei, and S. J. Ashtiani, "Enhancing Q-learning via state-space design for active battery balancing," Battery Energy, vol. 5, pp. 1–17, 2026, doi: https://doi.org/10.1002/bte2.70084.
[5] A. Filipović, "The role of artificial intelligence in video game development," Kultura Polisa, 2023, doi: https://doi.org/10.51738/kpolisa2023.20.3r.50f.
[6] N. Golіan, V. Golіan, and I. Afanasieva, "Black and white-box unit testing for web applications," Bulletin of NTU "KhPI." Series: System Analysis, Control and Information Technologies, 2022, doi: https://doi.org/10.20998/2079-0023.2022.01.13.
[7] S. Ibrahim, M. Mostafa, A. Jnadi, H. Salloum, and P. Osinenko, "Comprehensive overview of reward engineering and shaping in advancing reinforcement learning applications," IEEE Access, vol. 12, pp. 175473–175500, 2024, doi: https://doi.org/10.1109/access.2024.3504735.
[8] D. Jagdale, "Finite state machine in game development," Int. J. Adv. Res. Sci. Commun. Technol., 2021, doi: https://doi.org/10.48175/ijarsct-2062.
[9] S. Jarzabek and K. Kumar, "On interplay between separation of concerns and genericity principles: Beyond code weaving," Comput. Sci. Inf. Syst., vol. 13, pp. 731–758, 2016, doi: 10.2298/csis160129028j.
[10] P. Ladosz, L. Weng, M. Kim, and H. Oh, "Exploration in deep reinforcement learning: A survey," Inf. Fusion, 2022, doi: https://doi.org/10.1016/j.inffus.2022.03.003.
[11] S. Lapeyrade, "Reasoning with ontologies for non-player character's decision-making in games," Proc. AAAI Conf. Artif. Intell. Interact. Digit. Entertain., vol. 18, pp. 303–306, 2022, doi: https://doi.org/10.1609/aiide.v18i1.21980.
[12] H. Liu, J. Cao, and Z. Wang, "Research on path planning of mobile robot in complex environment," Discov. Appl. Sci., vol. 7, 2025, doi: https://doi.org/10.1007/s42452-025-06713-y.
[13] F. Luo, T. Xu, H. Lai, X.-H. Chen, W. Zhang, and Y. Yu, "A survey on model-based reinforcement learning," Sci. China Inf. Sci., vol. 67, 2022, doi: https://doi.org/10.1007/s11432-022-3696-5.
[14] Y. Lyu, A. Côme, Y. Zhang, and M. S. Talebi, "Scaling up Q-learning via exploiting state–action equivalence," Entropy, vol. 25, 2023, doi: https://doi.org/10.3390/e25040584.
[15] N. Mehanna and W. Rudametkin, "Caught in the game: On the history and evolution of web browser gaming," in Companion Proc. ACM Web Conf. 2023, 2023, doi: https://doi.org/10.1145/3543873.3585572.
[16] N. Midzic and M. Novak, "Performance comparison of WebAssembly and Phaser in procedural maze generation," Eurasia Proc. Sci. Technol. Eng. Math., 2025, doi: https://doi.org/10.55549/epstem.1753850.
[17] S. J, "Improving non-playable-characters (NPC) in games using reinforcement learning," Int. J. Res. Appl. Sci. Eng. Technol., 2025, doi: https://doi.org/10.22214/ijraset.2025.76473.
[18] S. Paul and J. Deshmukh, "Multi agent path finding using evolutionary game theory," arXiv:2212.02010, 2022, doi: 10.48550/arxiv.2212.02010.
[19] H. N and P. A. G, "A brief study of deep reinforcement learning with epsilon-greedy exploration," Int. J. Comput. Digit. Syst., 2022, doi: https://doi.org/10.12785/ijcds/110144.
[20] A. Qaffas, "An operational study of video games' genres," Int. J. Interact. Mob. Technol., vol. 14, pp. 175–194, 2020, doi: https://doi.org/10.3991/ijim.v14i15.16691.
[21] A. Rahayu, "Metode penelitian dan pengembangan (R&D): Pengertian, jenis dan tahapan," DIAJAR: J. Pendidikan Pembelajaran, 2025, doi: https://doi.org/10.54259/diajar.v4i3.5092.
[22] R. Ramadan and Y. Widyani, "Game development life cycle guidelines," in 2013 Int. Conf. Adv. Comput. Sci. Inf. Syst. (ICACSIS), 2013, pp. 95–100, doi: https://doi.org/10.1109/icacsis.2013.6761558.
[23] C. Shi, R. Wan, R. Song, W. Lu, and L. Leng, "Does the Markov decision process fit the data: Testing for the Markov property in sequential decision making," in Proc. 37th Int. Conf. Mach. Learn., 2020, pp. 8807–8817, doi: https://doi.org/10.48550/arxiv.2002.01751.
[24] J. Stenros, "The game definition game," Games Cult., vol. 12, pp. 499–520, 2016, doi: https://doi.org/10.1177/1555412016655679.
[25] K. Ter, O.-O. Ajayi, and D. Udekwe, "Taxonomy and trends in reinforcement learning for robotics and control systems: A structured review," arXiv:2510.21758, 2025, doi: https://doi.org/10.48550/arxiv.2510.21758.
[26] Y. Wen, S. Li, M. Fu, Y. Yang, X. Wang, and P. Liu, "Curriculum reinforcement learning with measurable task representation learning," Neural Netw., vol. 202, p. 109019, 2026, doi: https://doi.org/10.1016/j.neunet.2026.109019.
[27] V. Zangirolami and M. Borrotti, "Dealing with uncertainty: Balancing exploration and exploitation in deep recurrent reinforcement learning," Knowl.-Based Syst., vol. 293, p. 111663, 2023, doi: https://doi.org/10.1016/j.knosys.2024.111663.
[28] J. Zhang, Z. Rong, G. Zheng, J. Zhang, and L. Chen, "The emergence of cooperation via Q-learning in spatial donation game," J. Phys. Complex., vol. 5, 2024, doi: https://doi.org/10.1088/2632-072x/ad3f65.
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Muhammad Agil Perdana, Rasim, Rizky Rahman J.
This work is licensed under a Creative Commons Attribution 4.0 International License.
