analiza aiqbench
This paper introduces the Controlled Dynamics Attractor Transformer (CDAT), which integrates associative memory principles into transformer architectures to achieve biologically plausible, continuous-time inference dynamics. By coupling attention energy landscapes with Hopfield refinement and CANN-inspired modulation, CDAT establishes a topology-constrained dynamical system that improves performance on graph anomaly detection and classification tasks. The work represents significant progress in merging modern self-attention mechanisms with the stable retrieval properties of attractor networks for structured data processing.
Controlled Dynamics Attractor Transformer
arXiv:2606.15207v1 Announce Type: new Abstract: Transformer architectures have dramatically advanced representation learning and inference in deep models through self-attention mechanisms. In parallel,associative memory (AM) frameworks map representations onto energy landscapes, offering interpretable retrieval mechanisms. However, their continuous-time inference dynamics lack the biological plausibility of classical Continuous Attractor Neural Networks (CANNs). To bridge this gap, we propose Controlled Dynamics Attractor Transformer (CDAT), which couples a mixture von Mises-Fisher (Mo-vMF) attention energy with a Hopfield refinement energy, while augmenting energy descent with a CANN-inspired excitation-inhibition modulation. CDAT instantiates a topology-constrained dynamical system whose couplings encode relational structure among tokens, thereby linking attractor-style dynamics to modern energy-based attention. We further provide a constructive dissipation analysis to formally establish their controlled inference dynamics. Benefiting from these robust and structured dynamics, CDAT achieves state-of-the-art performance across multiple benchmarks in graph anomaly detection and graph classification.
Abstract:Transformer architectures have dramatically advanced representation learning and inference in deep models through self-attention mechanisms. In parallel,associative memory (AM) frameworks map representations onto energy landscapes, offering interpretable retrieval mechanisms. However, their continuous-time inference dynamics lack the biological plausibility of classical Continuous Attractor Neural Networks (CANNs). To bridge this gap, we propose Controlled Dynamics Attractor Transformer (CDAT), which couples a mixture von Mises-Fisher (Mo-vMF) attention energy with a Hopfield refinement energy, while augmenting energy descent with a CANN-inspired excitation-inhibition modulation. CDAT instantiates a topology-constrained dynamical system whose couplings encode relational structure among tokens, thereby linking attractor-style dynamics to modern energy-based attention. We further provide a constructive dissipation analysis to formally establish their controlled inference dynamics. Benefiting from these robust and structured dynamics, CDAT achieves state-of-the-art performance across multiple benchmarks in graph anomaly detection and graph classification.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
arXiv:2606.15207 [cs.LG]
(or arXiv:2606.15207v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.15207 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 237 cuvinte
Subiecte conexe
Learning Chaos Without Seeing Chaos: Extrapolation of Global Dynamics in Autoregressive Transformers
arXiv · 1 oct. 2026
Trajectory Geometry of Transformer Representations Across Layers
arXiv · 12 iun. 2026
Trajectory Geometry of Transformer Representations Across Layers
arXiv · 9 iun. 2026
Graph Neural Network-Based Reinforcement Learning for Controlling Biological Networks - the GATTACA Framework
arXiv · 15 oct. 2026