aiqbench analysis
This paper proposes a post-training method that compresses the fixed-size recurrent state of Mamba-2 models by truncating along a learned write-read subspace (via balanced truncation) rather than individual state coordinates, plus a per-layer rank allocation scheme (HARP). Reported results are a 1.8% WikiText-2 perplexity increase when halving the state dimension across 130M to 2.7B models, versus 9.3% for the GHOST baseline, with a single-pass calibration. For readers working on efficient sequence models, the main takeaways are the decoding-memory gains and the claim that coordinate-wise scoring misses the directions that matter, though the results depend on the model sizes and benchmarks tested.
Balanced State Subspaces for Mamba-2 State Compression
arXiv:2610.36617v1 Announce Type: cross Abstract: Mamba-2 compresses all context into a fixed-size recurrent state that dominates decoding memory and bandwidth. Existing compressors quantize state values or prune state coordinates, scoring each channel independently in the native basis. We examine trained Mamba-2 states and find that energy is almost rank-one, but the output does not read the energetic directions: energy concentrates in a bias-like direction accumulated by the recurrence, while the content the output reads lives in a low-dimensional subspace that is not aligned with the state's coordinates. Even output-aware scores are computed coordinate by coordinate, so they miss the directions that determine the input--output map. Compression should therefore target the write--read subspace rather than individual coordinates. We propose \method{}, a post-training transform that instantiates balanced truncation with full controllability and observability Gramians and applies the rotation exactly via Mamba-2's scalar-identity transition. Because the useful rank varies widely across layers, we further introduce HARP, which allocates ranks per layer from the Hankel spectrum and protects retrieval-sensitive layers, improving both perplexity and retrieval over uniform budgets. Calibration takes a single forward pass, an order of magnitude faster than GHOST's released protocol, with lower peak memory. With HARP, \method{} halves the state dimension at an average WikiText-2 perplexity increase of $0.3$ points ($1.8\%$) across 130M--2.7B, versus $1.6$ ($9.3\%$) for GHOST, and the gap widens at 25\% retention.
Abstract:Mamba-2 compresses all context into a fixed-size recurrent state that dominates decoding memory and bandwidth. Existing compressors quantize state values or prune state coordinates, scoring each channel independently in the native basis. We examine trained Mamba-2 states and find that energy is almost rank-one, but the output does not read the energetic directions: energy concentrates in a bias-like direction accumulated by the recurrence, while the content the output reads lives in a low-dimensional subspace that is not aligned with the state's coordinates. Even output-aware scores are computed coordinate by coordinate, so they miss the directions that determine the input--output map. Compression should therefore target the write--read subspace rather than individual coordinates. We propose \method{}, a post-training transform that instantiates balanced truncation with full controllability and observability Gramians and applies the rotation exactly via Mamba-2's scalar-identity transition. Because the useful rank varies widely across layers, we further introduce HARP, which allocates ranks per layer from the Hankel spectrum and protects retrieval-sensitive layers, improving both perplexity and retrieval over uniform budgets. Calibration takes a single forward pass, an order of magnitude faster than GHOST's released protocol, with lower peak memory. With HARP, \method{} halves the state dimension at an average WikiText-2 perplexity increase of $0.3$ points ($1.8\%$) across 130M--2.7B, versus $1.6$ ($9.3\%$) for GHOST, and the gap widens at 25\% retention.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Applications (stat.AP); Methodology (stat.ME)
arXiv:2610.36617 [cs.LG]
(or arXiv:2610.36617v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.36617 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 324 words
Related coverage
GS-MoE: Training-Free MoE Compression via Groupwise Subspace Sharing and Pruning
arXiv · Oct 27, 2026
Compressed Active Subspaces for Scalable Bayesian Inference
arXiv · Sep 18, 2026
Researchers Link Code Subspaces to Preserving Quantum Measurements
Quantum Zeitgeist · Nov 1, 2026
Subspace Crystallization in Grokking: Rank Compression Tracks the Phase Transition
arXiv · Oct 27, 2026