analiza aiqbench
The Platonic Transformer introduces a novel attention mechanism that incorporates geometric symmetries without incurring the computational overhead typical of existing equivariant methods. By maintaining the architecture and efficiency of standard Transformers while achieving competitive performance across computer vision and molecular prediction benchmarks, this approach offers a practical solution for integrating geometric inductive biases into scalable models.
Platonic Transformers: A Solid Choice For Equivariance
arXiv:2510.03511v3 Announce Type: replace-cross Abstract: While widespread, Transformers lack inductive biases for geometric symmetries common in science and computer vision. Existing equivariant methods often sacrifice the efficiency and flexibility that make Transformers so effective through complex, computationally intensive designs. We introduce the Platonic Transformer to resolve this trade-off. By defining attention relative to reference frames from the Platonic solid symmetry groups, our method induces a principled weight-sharing scheme. This enables combined equivariance to continuous translations and Platonic symmetries, while preserving the exact architecture and computational cost of a standard Transformer. Furthermore, we show that this attention is formally equivalent to a dynamic group convolution, which reveals that the model learns adaptive geometric filters and enables a highly scalable, linear-time convolutional variant. Across diverse benchmarks in computer vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25), the Platonic Transformer achieves competitive performance by leveraging these geometric constraints at no additional cost.
Abstract:While widespread, Transformers lack inductive biases for geometric symmetries common in science and computer vision. Existing equivariant methods often sacrifice the efficiency and flexibility that make Transformers so effective through complex, computationally intensive designs. We introduce the Platonic Transformer to resolve this trade-off. By defining attention relative to reference frames from the Platonic solid symmetry groups, our method induces a principled weight-sharing scheme. This enables combined equivariance to continuous translations and Platonic symmetries, while preserving the exact architecture and computational cost of a standard Transformer. Furthermore, we show that this attention is formally equivalent to a dynamic group convolution, which reveals that the model learns adaptive geometric filters and enables a highly scalable, linear-time convolutional variant. Across diverse benchmarks in computer vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25), the Platonic Transformer achieves competitive performance by leveraging these geometric constraints at no additional cost.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
arXiv:2510.03511 [cs.CV]
(or arXiv:2510.03511v3 [cs.CV] for this version)
Text integral sub licența deschisă a publicației · 173 cuvinte
Subiecte conexe
Equivariant Music Transformer
arXiv · 5 aug. 2026
Moment kernels: a simple and scalable approach for equivariance to rotations and reflections in deep convolutional networks
arXiv · 3 aug. 2026
Measuring Learned Equivariance: An Information Theoretic Approach
arXiv · 15 oct. 2026
REViT-v2: Hierarchical Windowed Roto-reflection Equivariant ViT for Equivariant Feature Extraction
arXiv · 7 oct. 2026