Survey On Designing ML Agents

Abstract: Artificial intelligence offers valuable methods for crafting complex problem-solving scenarios, with recent advancements allowing the development of agents capable of human-level or even superhuman performance. Reinforcement learning (RL), particularly through tools like the Unity ML-Agents toolkit, enables developers to incorporate machine learning-driven behaviors into game environments without needing specialized expertise. This paper reviews and compares various reinforcement learning techniques, detailing their application across two distinct training environments. We assess these methods in terms of training pace, generalization capabilities, and cumulative reward accumulation, with a focus on evaluating how combined extrinsic and intrinsic rewards influence training effectiveness in sparse reward settings. Our findings aim to support developers in selecting optimal reinforcement strategies to save time during training while enhancing performance and robustness. Results indicate that agents trained in sparse environments achieved faster progress with a mix of extrinsic and intrinsic rewards, while agents relying solely on extrinsic rewards struggled to complete tasks and exhibited suboptimal learning behaviors. Additionally, we discuss the role of exploration-exploitation trade-offs, curriculum learning, and reward shaping in improving agent performance.

Keywords: Unity, ML-Agents, Reinforcement Learning, Sparse Reward Environment, Artificial Intelligence, Machine Learning, Intrinsic Rewards, Extrinsic Rewards, Agent Training, Exploration-Exploitation, Curriculum Learning, Reward Shaping, Game Development, Autonomous Agents, Performance Evaluation, Generalization, Behavior Modeling, Policy Optimization.