aiqbench analysis
PRISM introduces a closed-form geometric bound that decomposes model degradation into scale, shape, and head components, offering a rigorous link between feature geometry and cross-entropy risk in post-training variants. By treating the shape term as a differentiable regularizer, this framework provides a novel mechanism to curb catastrophic forgetting more effectively than standard experience replay techniques. The method enables rapid ranking of quantized and fine-tuned models using only a single forward pass, addressing the critical need for efficient evaluation in an era of numerous LLM derivatives.
PRISM: A Geometric Risk Bound for Decomposing Drift into Scale, Shape, and Head
arXiv:2605.11608v2 Announce Type: replace Abstract: A single base LLM now comes with dozens of post-training variants, quantized, LoRA-adapted, or distilled, and each has to be checked before release. Existing evaluations provide only a partial picture: benchmark scores and likelihood screens say that a variant has degraded, similarity scores such as CKA and SVCCA say how its features moved, and nothing connects the two. We connect them with one structural fact and one design choice: the prediction head is linear, so feature geometry reaches the loss, and we compare the two feature sets through an orthogonal map, which leaves the geometry being measured unchanged. From these we derive PRISM, a closed-form upper bound on the cross-entropy risk gap between a target model and a proxy variant, and prove that it splits exactly into three measurable axes: scale, shape, and head. Each axis names a failure mode and where to intervene: low-bit quantization distorts shape and, at the lowest bit-widths, also inflates activation scale; quantizing the output embedding inflates the head term, which dominates the bound at high bit-widths. Because the shape term is differentiable, the same geometry becomes a regularizer that curbs catastrophic forgetting more than experience replay. Across two model families and five benchmarks, PRISM ranks quantized and fine-tuned variants from a single forward pass at mean Spearman above 0.8, and eight reference sequences already recover the ranking that the risk gap itself needs over a hundred to reach.
Abstract:A single base LLM now comes with dozens of post-training variants, quantized, LoRA-adapted, or distilled, and each has to be checked before release. Existing evaluations provide only a partial picture: benchmark scores and likelihood screens say that a variant has degraded, similarity scores such as CKA and SVCCA say how its features moved, and nothing connects the two. We connect them with one structural fact and one design choice: the prediction head is linear, so feature geometry reaches the loss, and we compare the two feature sets through an orthogonal map, which leaves the geometry being measured unchanged. From these we derive PRISM, a closed-form upper bound on the cross-entropy risk gap between a target model and a proxy variant, and prove that it splits exactly into three measurable axes: scale, shape, and head. Each axis names a failure mode and where to intervene: low-bit quantization distorts shape and, at the lowest bit-widths, also inflates activation scale; quantizing the output embedding inflates the head term, which dominates the bound at high bit-widths. Because the shape term is differentiable, the same geometry becomes a regularizer that curbs catastrophic forgetting more than experience replay. Across two model families and five benchmarks, PRISM ranks quantized and fine-tuned variants from a single forward pass at mean Spearman above 0.8, and eight reference sequences already recover the ranking that the risk gap itself needs over a hundred to reach.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2605.11608 [cs.CL]
(or arXiv:2605.11608v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.11608 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 333 words
Related coverage
Geometry Adaptive Counterfactual Distribution Learning with Diffusion-Guided Smoothing
arXiv · Jul 21, 2026
Deconfusing Corrigibility
AI Alignment Forum · Oct 17, 2026
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
BAIR Blog · May 8, 2026
Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems
arXiv · Sep 3, 2026