When Is Availability-Aware Training Worth It? A Benchmark and Empirical Study of Interruption-Resilient Optimization Under Predictable Compute Schedules

Title:When Is Availability-Aware Training Worth It? A Benchmark and Empirical Study of Interruption-Resilient Optimization Under Predictable Compute Schedules

Abstract:Training under non-stationary but predictable compute availability (satellites under eclipse, duty-cycled edge devices, power-capped datacenters) is often framed as needing specialized, availability-aware optimizers. We test that premise. We release OrbitTrace, a benchmark of 50 physics-grounded availability traces from SGP4 propagation of live two-line element sets across three orbital regimes, and ask a falsifiable question: when an availability gap interrupts training, is a specialized resumption strategy worth it, or is competent checkpoint-and-resume enough? Our central finding: when optimizer state can be preserved across a gap, the gap is essentially free. A strong checkpoint baseline that restores full optimizer state and indexes its learning-rate schedule in effective (active) time matches uninterrupted training to within data-ordering noise on CIFAR-10/ResNet-18, and exactly on a GPT-2/AdamW task. Advantages previously reported for availability-aware methods, including our own three-pillar method AAT, arise almost entirely from comparison against a weak baseline that indexes its schedule on wall-clock time. Against the strong baseline, reactive adaptation provides no advantage across stationary, optimizer-state-loss, and distribution-drift regimes. We isolate one narrow regime where it helps: for large models whose optimizer state cannot be persisted across gaps and that are interrupted by frequent, short pauses, reconstructing a decayed optimizer moment recovers only ~21% of the state-loss penalty on average (and not robustly across seeds); this vanishes for eclipse-scale gaps, where the decayed moment is indistinguishable from zero. Our contributions are a benchmark, a strong reproducible baseline protocol, and a clear characterization of when interruption-resilient optimization is worth its complexity, and when it is not.

Signal Processing (eess.SP); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)

arXiv:2609.22087 [eess.SP]

(or arXiv:2609.22087v1 [eess.SP] for this version)

https://doi.org/10.48550/arXiv.2609.22087 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.