aiqbench analysis
This paper presents the Complex State Propagator (CSP), a minimalist recurrent architecture designed specifically for deterministic state tracking tasks like parity checking and modular counting, demonstrating that complex-valued state propagation alone can outperform attention-heavy Transformers on these specific problems. By utilizing input-dependent rotations in the complex domain alongside block-level skip connections to prevent gradient degradation, CSP achieves perfect accuracy without relying on intermediate output projections or standard self-attention mechanisms. The work highlights a significant efficiency gain for specialized sequence modeling scenarios where full transformer expressivity is unnecessary overhead.
State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking
arXiv:2608.03425v1 Announce Type: new Abstract: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms. However, for a class of deterministic state tracking tasks---such as parity checking, modular counting, and parenthesis matching---attention may be overkill. In this paper, we show that \textbf{state propagation alone is sufficient}. We propose the \textbf{Complex State Propagator (CSP)}, a minimalistic recurrent architecture that \textbf{only propagates hidden states} across layers without output projections at intermediate steps. The state is represented as a complex-valued vector, updated via input-dependent rotations in the complex domain. To enable deep propagation without gradient vanishing or degradation, we introduce a \textbf{block-level skip connection} alongside element-wise complex normalization and SiLU activation at sequence boundaries. Applied with Focal Loss, CSP achieves \textbf{100\% accuracy} with perfect F1 scores across canonical tasks.
Abstract:Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms. However, for a class of deterministic state tracking tasks---such as parity checking, modular counting, and parenthesis matching---attention may be overkill. In this paper, we show that \textbf{state propagation alone is sufficient}. We propose the \textbf{Complex State Propagator (CSP)}, a minimalistic recurrent architecture that \textbf{only propagates hidden states} across layers without output projections at intermediate steps. The state is represented as a complex-valued vector, updated via input-dependent rotations in the complex domain. To enable deep propagation without gradient vanishing or degradation, we introduce a \textbf{block-level skip connection} alongside element-wise complex normalization and SiLU activation at sequence boundaries. Applied with Focal Loss, CSP achieves \textbf{100\% accuracy} with perfect F1 scores across canonical tasks.
Artificial Intelligence (cs.AI)
arXiv:2608.03425 [cs.AI]
(or arXiv:2608.03425v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.03425 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 220 words
Related coverage
Game Design Creativity evals
LessWrong · Oct 12, 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
Notes on Algorithms
LessWrong · Jun 10, 2026