analiza aiqbench
LieBN advances the frontier of geometric deep learning by introducing a unified framework for batch normalization over Lie groups and specific manifolds like SPD matrices. By leveraging invariant metrics to control Riemannian mean and variance, this method addresses critical limitations in existing approaches that are often restricted to single geometries or lack theoretical guarantees. The work demonstrates improved stability across nine distinct manifold structures, offering researchers a robust tool for handling complex data distributions beyond standard Euclidean spaces.
LieBN: Batch Normalization over Lie Groups
arXiv:2607.08783v3 Announce Type: replace-cross Abstract: Manifold-valued measurements are prevalent in various machine learning tasks. Recent advances have extended Deep Neural Networks (DNNs) to operate on manifolds. These extensions have been accompanied by normalization techniques tailored to different geometries, collectively referred to as Riemannian normalization. However, most existing Riemannian normalization methods are either designed for specific manifolds or fail to effectively normalize manifold-valued sample distributions. To address these limitations, we propose LieBN, a framework for Riemannian Batch Normalization (RBN) over Lie groups. Our approach leverages the theoretically convenient left- and right-invariant metrics, which naturally exist in every Lie group, and provides theoretical guarantees for controlling the Riemannian mean and variance. We instantiate LieBN across nine distinct geometries: four on the Symmetric Positive Definite (SPD) manifold, one on the group of rotation matrices, and four on the manifold of full-rank correlation matrices. Notably, among the SPD metrics, we introduce a novel right-invariant metric and extend three existing Lie group structures via matrix power deformation. Experiments on different manifolds validate the effectiveness of our framework. The code is available at https://github.com/GitZH-Chen/LieBN.git.
Abstract:Manifold-valued measurements are prevalent in various machine learning tasks. Recent advances have extended Deep Neural Networks (DNNs) to operate on manifolds. These extensions have been accompanied by normalization techniques tailored to different geometries, collectively referred to as Riemannian normalization. However, most existing Riemannian normalization methods are either designed for specific manifolds or fail to effectively normalize manifold-valued sample distributions. To address these limitations, we propose LieBN, a framework for Riemannian Batch Normalization (RBN) over Lie groups. Our approach leverages the theoretically convenient left- and right-invariant metrics, which naturally exist in every Lie group, and provides theoretical guarantees for controlling the Riemannian mean and variance. We instantiate LieBN across nine distinct geometries: four on the Symmetric Positive Definite (SPD) manifold, one on the group of rotation matrices, and four on the manifold of full-rank correlation matrices. Notably, among the SPD metrics, we introduce a novel right-invariant metric and extend three existing Lie group structures via matrix power deformation. Experiments on different manifolds validate the effectiveness of our framework. The code is available at this https URL.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2607.08783 [cs.LG]
(or arXiv:2607.08783v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 189 cuvinte
Subiecte conexe
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta Engineering · 3 aug. 2026
Debugging Logprob Mismatches in LLM Reinforcement Learning
AMD · 23 sept. 2026
Thread Trace Part 1: ROCprof Compute Viewer
AMD · 15 sept. 2026
Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference
NVIDIA · 31 iul. 2026