Towards Sustainable Hydrogen Systems: Supply Chain Optimization with Model Predictive Control and Reinforcement Learning

Abstract

Hydrogen supply chains are expected to play a central role in future low-carbon energy systems. However, the operation of such systems is inherently challenged by factors such as uncertainty in renewable availability, electricity prices, and hydrogen demand, as well as by practical engineering constraints associated with electrolyzers, energy storage, and grid interaction. Effective control strategies must therefore balance economic performance, operational feasibility, and sustainability objectives under time-varying and uncertain conditions, especially as hydrogen infrastructure scales beyond pilot deployments.

This paper investigates and compares four operational control approaches for a renewable-powered hydrogen supply chain: a rule-based controller (RBC), model predictive control (MPC), reinforcement learning without forecasts (RL-NF), and reinforcement learning with forecast-augmented observations (RL-F). All methods are evaluated within a unified, physically realistic framework that incorporates electrolyzer minimum load and ramping constraints, battery and hydrogen storage dynamics, grid import limits, renewable curtailment, and consistent economic assumptions. This unified formulation ensures methodological consistency, enabling a fair and rigorous comparison of predictive and adaptive control paradigms under identical system conditions and uncertainty realizations.

Simulation results show that MPC achieves the highest economic performance by effectively exploiting short-term forecasts to reduce grid dependence and optimize storage utilization, highlighting the value of predictive optimization when reliable forecasts are available. RL-NF demonstrates robust and competitive performance despite operating without future information, while RL-F does not consistently outperform its no-forecast counterpart, illustrating the challenges of learning from noisy forecasts in complex energy systems. These findings illustrate the relative strengths and limitations of predictive and learning-based control approaches for hydrogen supply chains under uncertainty and motivate further investigation into their roles in large-scale hydrogen system operation.