A Theoretical Sim-to-Real Transfer Framework for Deep Reinforcement Learning in Autonomous Spacecraft Attitude Control

Abstract

Deep Reinforcement Learning (DRL) has demonstrated promising performance for spacecraft attitude control in simulation environments. However, the transition from simulation-trained policies to real spacecraft operation remains limited due to modeling uncertainties, safety constraints, and certification requirements. This paper proposes a structured theoretical Sim-to-Real transfer framework for DRL-based spacecraft Attitude Determination and Control Systems (ADCS).

The spacecraft's rotational dynamics are formulated as a nonlinear control system, and the learning objective is defined under bounded control and safety constraints. The proposed architecture integrates seven coordinated modules, including nominal dynamic modeling, domain randomization, sensor and actuator realism modeling, robustness-oriented policy training, a Safety and Certification Envelope, bounded post-deployment adaptation, and onboard runtime supervision. The Safety Envelope is formalized as a constrained control set with projection-based action filtering, while parameter adaptation is restricted within bounded update limits to preserve certification assumptions.

The resulting system can be interpreted as a constrained adaptive nonlinear closed-loop control architecture that ensures bounded-input bounded-state behavior under operational limits. Rather than introducing a new learning algorithm, this work contributes a certification-aware architectural framework that bridges reinforcement learning and aerospace assurance principles. The proposed design establishes a rigorous foundation for future simulation-based, hardware-in-the-loop, and operational validation of learning-enabled spacecraft autonomy.