aiqbench analysis
This research introduces Geometric Evolution Maps (GEMs) to address the instability of concept probes in Transformer residual streams, demonstrating that concept directions rotate substantially during formation and only settle after a specific Concept Allocation Zone. By tracking these trajectories across multiple segments rather than relying on fixed layers or rotation criteria, the method provides more precise and causal ablation targets for understanding model internals. The study significantly refines its claims based on an expanded corpus of 29 models, correcting previous artifacts regarding scale floors and architectural splits to establish a robust framework for probing deep learning representations.
Geometric Evolution Maps: Extracting Stable Concept Probes from Transformer Residual Streams
arXiv:2605.25848v2 Announce Type: replace Abstract: A concept probe is only as reliable as the layer it is taken from. Probing at a fixed late layer, or at the peak of a separation curve, ignores a structural feature of how concepts form: the probe direction rotates substantially during assembly and does not settle until after the Concept Allocation Zone (CAZ) in which it forms. We introduce Geometric Evolution Maps (GEMs). A GEM records a concept's directional trajectory across one CAZ segment, takes the settled direction at that segment's final layer, and locates the handoff layer immediately beyond it - the first layer at which that direction can be evaluated outside the window it was estimated on. The handoff layer is derived from the segment boundary, not detected by a rotation criterion. A concept typically occupies several segments; we call that set its atlas. This rotation is substantial and consistent across architectures and concept types - a property of the representation, not an estimation artifact. The extracted direction is causal: ablating it suppresses far more separation than ablating a random direction, though no single site carries the effect alone. Probing after rotation completes is more precise than probing during it; a depth-matched control shows this advantage comes from probing at greater depth, not any privilege specific to the handoff boundary. A small number of structured exceptions are documented rather than left unaccounted for. A concept's atlas - its full set of GEMs, not any single one of them - is therefore the unit this method delivers for downstream use. GEM provides the ablation-targeting methodology used by the CAZ Validation paper in this program.
Abstract:A concept probe is only as reliable as the layer it is taken from. Probing at a fixed late layer, or at the peak of a separation curve, ignores a structural feature of how concepts form: the probe direction rotates substantially during assembly and does not settle until after the Concept Allocation Zone (CAZ) in which it forms. We introduce Geometric Evolution Maps (GEMs). A GEM records a concept's directional trajectory across one CAZ segment, takes the settled direction at that segment's final layer, and locates the handoff layer immediately beyond it - the first layer at which that direction can be evaluated outside the window it was estimated on. The handoff layer is derived from the segment boundary, not detected by a rotation criterion. A concept typically occupies several segments; we call that set its atlas. This rotation is substantial and consistent across architectures and concept types - a property of the representation, not an estimation artifact. The extracted direction is causal: ablating it suppresses far more separation than ablating a random direction, though no single site carries the effect alone. Probing after rotation completes is more precise than probing during it; a depth-matched control shows this advantage comes from probing at greater depth, not any privilege specific to the handoff boundary. A small number of structured exceptions are documented rather than left unaccounted for. A concept's atlas - its full set of GEMs, not any single one of them - is therefore the unit this method delivers for downstream use. GEM provides the ablation-targeting methodology used by the CAZ Validation paper in this program.
Comment: v2: substantial revision. Corpus grown from 23 to 29 base models. Several v1 claims withdrawn, most notably the MHA/GQA cohort split and the sub-500M scale-floor account. The depth-matched settling control is replaced by a corrected version that reverses its conclusion. See Appendix E for the full changelist
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2605.25848 [cs.LG]
(or arXiv:2605.25848v2 [cs.LG] for this version)
Full text under the publisher's open licence · 328 words
Related coverage
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · Oct 8, 2026
Large to Small Model Stitching Destroys What You Built It to Carry Along the Fit
LessWrong · Oct 11, 2026
Equation-Free Period-Aware Forecast-Error Contraction for Estimating Negative Largest Lyapunov Exponents from Short Trajectory Ensembles
arXiv · Aug 7, 2026
Researchers map quantum engine errors with a new topological tool
Quantum Zeitgeist · Aug 22, 2026