Multi-Modal Temporal Learning for Personalized Stroke Balance Function Recovery Prediction and Treatment Allocation

Abstract

Stroke causes long-term disability, impacting balance and requiring personalized rehabilitation. Existing methods face resource constraints and outcome variability, hindering accurate recovery prediction and treatment allocation. We propose STarNet (Stroke Temporal-aware Recovery Network), a novel multi-modal temporal learning framework for personalized stroke balance recovery prediction and treatment assignment. STarNet integrates diverse clinical, wearable IMU, and video time-series data via multi-modal encoders, individualized feature adaptation, and cross-modal fusion. It employs prediction heads for counterfactual outcome estimation and a novel loss for uncertainty quantification. Evaluated on a simulated dataset, STarNet achieved state-of-the-art performance, demonstrating superior accuracy in predicting Berg Balance Scale improvement and identifying treatment responders. Its personalized recommendations significantly outperformed baselines and clinicians in a simulated evaluation. Ablation studies confirmed component contributions and uncertainty reliability. STarNet offers a promising avenue for optimizing rehabilitation and individualizing post-stroke care, enhancing patient outcomes.