Action Response Is Not Factual Direction: A Prospective Audit of Robot Video World Models

Abstract

Action-conditioned video models are often called controllable when changing the action changes the rollout, but response magnitude alone establishes neither movement toward the recorded future nor reliable sample ranking. We present ActionProbe, an artifact-audited protocol that separates branch response, signed factual-video advantage, one fixed composite repeatability contrast, and best-of-two sample-selection utility. On 200 command-enriched windows from 40 episodes and four conditioner-held-out tasks, decoded-RGB response–advantage correlations are positive in two fixed task–donor cells and negative in two. Across three donors and decoded-RGB, DINOv2 semantic/spatial, and RAFT motion endpoints, the fold-stratified raw-scale advantage interval includes zero at every endpoint, whereas normalized RGB and RAFT advantages are small and positive; a post-hoc fixed-task-weight sensitivity further changes which lower bounds clear zero. Donor-to-repeat response ratios range from 0.505 to 0.744, with every episode-bootstrap interval below one, conditional on a mixed-campaign design whose repeat branch also changes seed. A frozen generator-residual selector likewise fails its prospective gate: fold gains are +0.350 and -0.062, and the 180-window diagnostic gain is 0.144 [-0.140,0.424]. Model-internal sensitivity should therefore not serve as factual-direction or sample-selection evidence without task-stratified, clustered, and prospective audit.