APF-MRPPO: Navigating Graph-Based Maps with APF and Multi-Reward Reinforcement Learning

Abstract

Mobile robot navigation in hazardous environments such as nuclear facilities, chemical warehouses, and earthquake-damaged buildings requires algorithms that balance global path planning with real-time hazard avoidance. Traditional approaches often fail under unexpected hazards, ambiguous obstacle cues, or dynamically changing environments. This work presents APF-MRPPO, a reinforcement learning method that integrates artificial potential fields (APF) with multi-reward proximal policy optimization (MRPPO) in a graph-based navigation framework. Using Unity and ROS2 with a Boston Dynamics Spot digital twin, we model hazards as repulsive forces and waypoints as attractive forces, embedding these into an environmental reward map. Experiments in simulation demonstrate that APF-MRPPO achieves 100% success with hazards covering 20% of waypoints, 85% success with 40% waypoint hazards, and 83% success with hazards three times larger than the robot. These results confirm the algorithm’s ability to adapt dynamically, even when random hazards occupy ~40% of the map. Current work focuses on tuning reward functions and enhancing potential field models, while future directions include integration of graph neural networks (GNNs) for context-aware path planning and deployment on physical Spot hardware. This study highlights the potential of combining reinforcement learning with potential field-based graph navigation for robust operation in complex, safety-critical environments