Decision making for lunar landing applications using AI agents and reinforcement learning.

Abstract

This study explores the decision making capabilities of large language model (LLM) artificial intelligence (AI) agents to automate learning in lunar landing missions. In particular, the work investigates the use of AI agents to minimise human intervention in training a lunar lander by providing high-level strategic guidance to a reinforcement learning (RL) agent within the complex simulation environment of Kerbal Space Program (KSP). To that end, LLM AI agents are utilised to interpret a lander manual, extract key information to construct the reward function of the RL algorithm, and dynamically refine it based on training performance. A comparative case study evaluates the effectiveness of GPT-3.5-Turbo, GPT-4, and Meta-Llama-3-70B, in optimising the RL training process for lunar landings. Extending this approach, the study further explores AI-assisted hyperparameter optimisation (HPO) to streamline RL training. Instead of relying on traditional, computationally expensive methods like grid search and Bayesian optimisation, a zero-shot tuning approach is introduced, where an AI agent configures RL hyperparameters from a single instruction prompt without iterative refinements. Using GPT-4o, this method is applied to four RL algorithms (DQN, PPO, A2C, and SAC Discrete), demonstrating significant improvements in training efficiency, convergence speed, and reduced human effort, particularly for SAC Discrete. These findings highlight the potential of LLMs to automate both reward function design and hyperparameter tuning, thus advancing AI capabilities in space exploration and autonomous navigation tasks.