Coarse-Graining Recursive Expert Computation: Continuation-Relative Interfaces and Failure Modes

Abstract:Computational coarse-graining aims to replace a sequence of updates with a cheaper module while preserving task performance. We present a continuation-relative evaluation framework that assesses replacements through the computation that follows them, using downstream task quality and total execution cost as joint criteria. The framework distinguishes boundary representations, summaries of skipped computation, executed operations, and downstream loss. It makes interfaces and permitted deployment information explicit, combines fine-path references with directly trained controls using the same shortened architecture, and accounts for selection overhead. An exact loss decomposition separates replacement effects at a fixed downstream action from effects of changing that action. A finite construction shows how retained information can reduce the stipulated cost of zero-error prediction under fixed information and controller restrictions. Experiments in two frozen pretrained language models show that representation fidelity and task loss can favor different fitting objectives. In a prespecified comparison on a 1.7-billion-parameter model, next-token-loss training lowers in-distribution negative log-likelihood by 0.1901 nats per scored token compared with reconstructing the skipped blocks' output, using the same optimizer and update count. This ranking reverses in all nine paired comparisons under distribution shift. In a recursive expert prototype, the jointly adapted candidate lowers task error relative to frozen-bank and router-only baselines, but trails directly trained controls, and the tested input-only selectors fail the joint quality-cost criteria. Together, the protocol and studies provide explicit criteria and diagnostics for distinguishing useful computational replacement from local approximation gains within specified interfaces and evaluation settings.

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computational Complexity (cs.CC); Computational Engineering, Finance, and Science (cs.CE)

arXiv:2610.14413 [cs.LG]

(or arXiv:2610.14413v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.14413 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

numerical-supplement/README.md

numerical-supplement/additional-results.md

numerical-supplement/factor_objective_effects.csv

numerical-supplement/factor_objective_effects.pdf

numerical-supplement/gpt2_complete_metrics.csv

numerical-supplement/gpt2_cont_ls_trajectory.csv

numerical-supplement/pretraining_quality.csv

numerical-supplement/pretraining_quality.pdf

numerical-supplement/qwen3_all_metrics.csv

numerical-supplement/realmodel_trajectory.pdf

numerical-supplement/reproduction-manifest.md

numerical-supplement/scientific-details.md

numerical-supplement/synthetic_family_quality.csv

(8 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.