aiqbench analysis
This paper proposes an algebraic framework that treats multiplication as a core architectural operation, aiming to unify convolutional, attention, state-space, and equivariant networks under one design principle. Its most concrete claims are that Mamba corresponds to a higher self-interaction order and that a simplified-algebra layer beats Mamba on CIFAR-10. Readers should weigh the unifying framing as a research hypothesis still needing broader benchmarks, though it may offer a useful lens for architecture design.
Product Interaction: An Algebraic Formalism for Deep Learning Architectures
arXiv:2602.02573v2 Announce Type: replace-cross Abstract: Modern neural architectures are largely developed as distinct empirical constructions, despite recurring principles across convolutional, attention, state-space, and equivariant models. We introduce Product Interactions, an algebraic formalism that unifies these architectures under a common construction principle and turns their design into a systematic procedure. The core shift is representing data as algebraic elements and treating multiplication, rather than linear transformation alone, as a primitive architectural operation. Architectures are generated by composing multiplication operators, with input dependence organized by self-interaction order. Within this formalism, we derive convolutional and equivariant networks as symmetry-constrained first-order product interactions, recover Mamba from state-space models as an increase in self-interaction order from one to three, and establish attention as a third-order interaction from composing the same operator. Equivariance is incorporated by constraining the algebraic product via symmetry: translation equivariance yields convolution, while SO(3)-compatibility produces the Clebsch--Gordan product and tensor-field interactions. This perspective recasts architecture design as an algebraic construction problem: the choice of algebra defines admissible interactions, symmetry constrains their structure, and composition generates higher-order architectures. The framework is thus generative. Following these principles, we construct a simplified-algebra dynamical layer that improves both accuracy and throughput over Mamba on CIFAR-10. Experiments show that interaction order affects accuracy non-monotonically and symmetry constraints boost learnability. Product Interactions provides a mathematical methodology for analyzing existing architectures and systematically designing new ones.
Abstract:Modern neural architectures are largely developed as distinct empirical constructions, despite recurring principles across convolutional, attention, state-space, and equivariant models. We introduce Product Interactions, an algebraic formalism that unifies these architectures under a common construction principle and turns their design into a systematic procedure. The core shift is representing data as algebraic elements and treating multiplication, rather than linear transformation alone, as a primitive architectural operation. Architectures are generated by composing multiplication operators, with input dependence organized by self-interaction order. Within this formalism, we derive convolutional and equivariant networks as symmetry-constrained first-order product interactions, recover Mamba from state-space models as an increase in self-interaction order from one to three, and establish attention as a third-order interaction from composing the same operator. Equivariance is incorporated by constraining the algebraic product via symmetry: translation equivariance yields convolution, while SO(3)-compatibility produces the Clebsch--Gordan product and tensor-field interactions. This perspective recasts architecture design as an algebraic construction problem: the choice of algebra defines admissible interactions, symmetry constrains their structure, and composition generates higher-order architectures. The framework is thus generative. Following these principles, we construct a simplified-algebra dynamical layer that improves both accuracy and throughput over Mamba on CIFAR-10. Experiments show that interaction order affects accuracy non-monotonically and symmetry constraints boost learnability. Product Interactions provides a mathematical methodology for analyzing existing architectures and systematically designing new ones.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2602.02573 [cs.LG]
(or arXiv:2602.02573v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2602.02573 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 317 words
