It Was the Learning Rate: Compute-Matched Baselines Erase the Apparent Gains of a Memristor Crossbar Repair Controller

Abstract

None of the memristor-crossbar repair papers we examined tests against a tuned, equal-compute baseline. We measured what that omission is worth. Our repair controller (D) paces its spare cells and gates retraining on a held-out set. It first looked 34.0 points ahead of a rival with living spares (a spare keeps training from the cell's current weight). Tuning the rival's learning rate cut the lead to 4.2 points. Giving it D's number of retraining (burst) steps cut it to 1.06. A pre-specified eight-seed replication cut it to -0.005 points (95% CI -1.7 to +1.7). An unfair baseline manufactured a 34-point win, and fair baselines removed it. D still beats retraining-only at equal burst steps by 3.78 points (5 of 5 seeds), so the controller is genuine where it holds. Two more findings: a forward-only BatchNorm refit stays at chance once spares run out (76.06 points below D, 0 of 8 seeds), and living spares match frozen-reference spares at a tuned rate (+0.03 points; an earlier +2.48 came from a barely working rate). The scope is simulation, ResNet-9 on CIFAR-10. The lesson is a call to the field: tuned, step-matched baselines and a replication are the price of a credible margin. All code, runs and analysis are public.