aiqbench analysis
SYNAPSE introduces a training-free framework for analyzing and stress-testing internal representations in Transformer models without altering the base architecture. By utilizing forward-hook interventions and lightweight linear probes, it reveals consistent organizational patterns of neurons while exposing vulnerabilities to small structured manipulations. This approach directly addresses the need for transparency and robustness in sensitive AI applications like healthcare and cybersecurity.
SYNAPSE: Framework for Neuron Analysis and Perturbation in Sequence Encoding
arXiv:2603.08424v2 Announce Type: replace Abstract: In recent years, Artificial Intelligence has become a powerful partner for complex tasks such as data analysis, prediction, and problem-solving, yet its lack of transparency raises concerns about its reliability. In sensitive domains such as healthcare or cybersecurity, ensuring transparency, trustworthiness, and robustness is essential, since the consequences of wrong decisions or successful attacks can be severe. To overcome these limitations, this work proposes SYNAPSE, a systematic, training-free framework (requiring no retraining of the base model) for understanding and stress-testing the internal behavior of Transformer models across domains. SYNAPSE extracts per-layer [CLS] representations, trains a lightweight linear probe to obtain global and per-class neuron rankings, and applies forward-hook interventions during inference. This design enables controlled, repeatable experiments on internal representations without altering the original model, thereby allowing weaknesses, stability patterns, and label-specific sensitivities to be measured and compared directly across tasks and architectures. SYNAPSE reveals a consistent organization of internal representations across the two evaluated domains, in which task-relevant information is encoded in broad, overlapping neuron subsets. This redundancy provides a strong degree of functional stability, while class-wise asymmetries expose heterogeneous specialization patterns and enable fine-grained, label-aware analysis. In contrast, small structured manipulations in weight or logit space are sufficient to redirect predictions, highlighting complementary vulnerability profiles and illustrating how SYNAPSE can guide the development of more robust and interpretable Transformer models.
Abstract:In recent years, Artificial Intelligence has become a powerful partner for complex tasks such as data analysis, prediction, and problem-solving, yet its lack of transparency raises concerns about its reliability. In sensitive domains such as healthcare or cybersecurity, ensuring transparency, trustworthiness, and robustness is essential, since the consequences of wrong decisions or successful attacks can be severe. To overcome these limitations, this work proposes SYNAPSE, a systematic, training-free framework (requiring no retraining of the base model) for understanding and stress-testing the internal behavior of Transformer models across domains. SYNAPSE extracts per-layer [CLS] representations, trains a lightweight linear probe to obtain global and per-class neuron rankings, and applies forward-hook interventions during inference. This design enables controlled, repeatable experiments on internal representations without altering the original model, thereby allowing weaknesses, stability patterns, and label-specific sensitivities to be measured and compared directly across tasks and architectures. SYNAPSE reveals a consistent organization of internal representations across the two evaluated domains, in which task-relevant information is encoded in broad, overlapping neuron subsets. This redundancy provides a strong degree of functional stability, while class-wise asymmetries expose heterogeneous specialization patterns and enable fine-grained, label-aware analysis. In contrast, small structured manipulations in weight or logit space are sufficient to redirect predictions, highlighting complementary vulnerability profiles and illustrating how SYNAPSE can guide the development of more robust and interpretable Transformer models.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2603.08424 [cs.LG]
(or arXiv:2603.08424v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.08424 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 317 words
Related coverage
Equal Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models
arXiv · Oct 10, 2026
Non-Perturbative Geometric Framework for Single-Qubit Gates under Always-On Couplings
arXiv · Jun 4, 2026
CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising
arXiv · Jul 31, 2026
Perturbative Variational Quantum Eigensolver via Reduced Density Matrices
arXiv · Aug 18, 2026