analiza aiqbench
This research introduces HySAT, a novel training method that applies hyperbolic geometry strictly to the loss layer rather than trainable adapters. The study demonstrates significant stability improvements across multiple expert SLMs by proving mathematically why this specific placement prevents gradient collapse in tree-structured domains. This advancement addresses critical architectural limitations of standard Euclidean transformers when handling hierarchical knowledge representation.
After the Euclidean Highway: Hyperbolic Expert AI as the Next Innovation
arXiv:2607.17513v1 Announce Type: cross Abstract: Expert domains are trees; the Euclidean transformer is not, diluting parent-child structure exponentially at depth. The hyperbolic turn left one question unasked: not how much of a network to curve, but where curvature may touch the gradient. Placement is a law, not a knob: the same geometry on a trainable adapter collapses training (seventeen training collapses, ~220 GPU-hours), yet at the loss layer alone it trains without one -- this is HySAT (Hyperbolic Structure-Aware Training), hyperbolic losses at the loss layer only. Across six expert SLMs we constructed and deployed (Llama 3.1 and EXAONE 3.5; four adapter strategies; 18.0M-sample corpus; zero NaN over ~317K optimizer steps), a matched four-arm ablation isolates the preserved manifold invariant, and three propositions and a lemma prove why loss-only placement is stable where adapter-on-manifold is not. Four models are operationally deployed (one live, consumer-facing), two open-weight, with per-step traces and a seventeen-incident failure ledger on Zenodo (CC-BY-4.0).
Abstract:Expert domains are trees; the Euclidean transformer is not, diluting parent-child structure exponentially at depth. The hyperbolic turn left one question unasked: not how much of a network to curve, but where curvature may touch the gradient. Placement is a law, not a knob: the same geometry on a trainable adapter collapses training (seventeen training collapses, ~220 GPU-hours), yet at the loss layer alone it trains without one -- this is HySAT (Hyperbolic Structure-Aware Training), hyperbolic losses at the loss layer only. Across six expert SLMs we constructed and deployed (Llama 3.1 and EXAONE 3.5; four adapter strategies; 18.0M-sample corpus; zero NaN over ~317K optimizer steps), a matched four-arm ablation isolates the preserved manifold invariant, and three propositions and a lemma prove why loss-only placement is stable where adapter-on-manifold is not. Four models are operationally deployed (one live, consumer-facing), two open-weight, with per-step traces and a seventeen-incident failure ledger on Zenodo (CC-BY-4.0).
40 pages, 11 figures. Supplementary Information included as an ancillary file. Data and code: Zenodo, concept DOI https://doi.org/10.5281/zenodo.21438499 (published, CC-BY-4.0)
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2607.17513 [cs.LG]
(or arXiv:2607.17513v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.17513 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 197 cuvinte
Subiecte conexe
New York Governor Kathy Hochul thinks AI should be ‘less evil’
The Verge · 31 aug. 2026
Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · 24 sept. 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026
New Eagle Raptor Innovation Summit 2026 to Explore Autonomy, Electrification and the Future of Intelligent Vehicles
roboticstomorrow.com · 3 sept. 2026