aiqbench analysis
This research presents a rigorous audit protocol for evaluating visual weight readers by distinguishing functional information from arbitrary parameter coordinates in neural networks. The study demonstrates how specific transformations, such as hidden-neuron permutations and ReLU rescalings, impact recognition accuracy and highlights the limitations of current evaluation metrics that fail to account for these coordinate shifts. By establishing an inspectable framework with exact-rational rounding bounds, the work advances the reliability of interpreting frozen language models and other neural architectures without relying on semantic cues alone.
Consistency Is Not Understanding: Orbit Audits for Neural Field Weight Readers
arXiv:2611.05798v1 Announce Type: new Abstract: Recognizing visual content from neural-network weights requires separating functional information from arbitrary parameter coordinates. We audit this separation with hidden-neuron permutations and positive ReLU rescalings that preserve the represented function. The study contains 480 independently fitted shape fields and 1,000 MNIST fields, with transformed copies grouped by origin. On shape recognition, a statistics reader falls from \ShapeRawStats\% to \ShapeJointStats\% accuracy under joint transformations; a normalized set reader retains \ShapeNormSet\% with unchanged predictions. Four small frozen language models obtain \LMRange\% raw accuracy with next-token scoring. A separate four-configuration reasoning experiment with 1,920 scored sessions obtains \FrontRawRange\% raw accuracy with generated answers, while fresh identical raw prompts disagree on \FrontRawRepeatRange\% of origins. Canonicalization makes equivalent prompts identical, but this neither establishes recognition nor guarantees identical generated responses. Joint correctness/agreement metrics, semantic-cue controls, fresh repeats, and exact-rational rounding bounds separate these failure modes. The result is an inspectable evaluation protocol for visual weight readers, with claims restricted to the tested fields, prompts, and execution configurations.
Abstract:Recognizing visual content from neural-network weights requires separating functional information from arbitrary parameter coordinates. We audit this separation with hidden-neuron permutations and positive ReLU rescalings that preserve the represented function. The study contains 480 independently fitted shape fields and 1,000 MNIST fields, with transformed copies grouped by origin. On shape recognition, a statistics reader falls from \ShapeRawStats\% to \ShapeJointStats\% accuracy under joint transformations; a normalized set reader retains \ShapeNormSet\% with unchanged predictions. Four small frozen language models obtain \LMRange\% raw accuracy with next-token scoring. A separate four-configuration reasoning experiment with 1,920 scored sessions obtains \FrontRawRange\% raw accuracy with generated answers, while fresh identical raw prompts disagree on \FrontRawRepeatRange\% of origins. Canonicalization makes equivalent prompts identical, but this neither establishes recognition nor guarantees identical generated responses. Joint correctness/agreement metrics, semantic-cue controls, fresh repeats, and exact-rational rounding bounds separate these failure modes. The result is an inspectable evaluation protocol for visual weight readers, with claims restricted to the tested fields, prompts, and execution configurations.
Artificial Intelligence (cs.AI)
arXiv:2611.05798 [cs.AI]
(or arXiv:2611.05798v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.05798 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 256 words
Related coverage
How to Use Parallel Data for Multilingual Pretraining
arXiv · Oct 22, 2026
Multi-level context Modeling for consistent expert selection in Mixture-of-Experts
arXiv · Jul 21, 2026
Independent Assessment of AI R&D Acceleration at Anthropic in October 2026
METR · Nov 5, 2026
MarKey: Marginal Utility Guided Greedy Keyframe Selection for Long Video Understanding
arXiv · Sep 15, 2026