aiqbench analysis
This research advances intrinsic explainability by modeling the temporal evolution of multilayer perceptrons within hyperbolic geometry, specifically using the Poincaré model to capture training trajectories. By constructing dynamic parameter graphs that account for neuron permutations over time, the study offers a novel framework for understanding how neural network structures self-organize during optimization. The findings provide deeper geometric insights into MLP behavior beyond static weight snapshots, potentially informing more robust metalearning architectures.
Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability
arXiv:2610.03000v1 Announce Type: cross Abstract: Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an optimization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our representation features the Poincar\'e model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints---as per established measure-based explainability methods---we construct temporal \textit{parameter graphs}, i.e., snapshots over time $T$ steps of the optimization/training process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based metalearning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincar\'e model ball, and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments on regression and classification tasks with trained MLPs show strong meta-network performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network's self-organization.
Abstract:Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an optimization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our representation features the Poincaré model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints---as per established measure-based explainability methods---we construct temporal \textit{parameter graphs}, i.e., snapshots over time $T$ steps of the optimization/training process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based metalearning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincaré model ball, and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments on regression and classification tasks with trained MLPs show strong meta-network performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network's self-organization.
Published as a main conference paper at ICLR 2026. 10 Main Pages, 22 pages of supplementary material
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Data Analysis, Statistics and Probability (physics.data-an)
arXiv:2610.03000 [cs.LG]
(or arXiv:2610.03000v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.03000 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 287 words
Related coverage
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026
Introducing PIRAMID: Physics-Informed Research for Ambitious Mechanistic Interpretability
LessWrong · Jul 25, 2026
Capsule Lens: Locating and Tracking Concept Geometry in Model Representations
arXiv · Sep 9, 2026
Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization
Meta Engineering · Jul 15, 2026