A Physics-Based Residual Ensemble Approach to Address the Limitations of GBDT Extrapolation

Abstract

Gradient Boosting Decision Trees (GBDT) are widely utilized in engineering for their efficiency on tabular data, yet they suffer from a fundamental architectural flaw: a structural inability to perform linear extrapolation, often resulting in dangerous 'flatlining' predictions in out-of-distribution (OOD) regimes. This paper introduces the Physics-Anchored Residual Blending (PARB) framework, a hybrid architecture designed to mitigate these limitations in thermal heat flux modeling. PARB decouples the prediction task into two stages: a Physics-Informed Tabular Transformer (PITT) backbone that captures global physical asymptotes, and a GBDT ensemble that refines material-specific empirical residuals.  

Our empirical validation using Finite Difference Method (FDM) simulations demonstrates that while pure GBDT models collapse to an R2 of 0.59 in extrapolation regions, PARB maintains a robust R2 of 0.85, representing a 44% improvement in predictive reliability. Furthermore, residual audits across 100 material profiles show superior stability in 96% of cases, reducing mean absolute error by ~63,000 W/m² with a minimal computational overhead of 1.15x. By enforcing physical invariants through origin-preserving MaxAbs scaling, PARB offers a scientifically grounded surrogate with potential applicability for real-time monitoring in high-energy thermal systems, providing a more reliable foundation for future integration into complex physical infrastructure.