analiza aiqbench
SPEAR introduces a novel architecture that decouples latent features into frequency components to mitigate knowledge interference in partial differential equation (PDE) modeling. By implementing a knowledge-guided expert aggregation strategy, the method reduces expert redundancy by 50% while enhancing transfer learning capabilities across heterogeneous dynamics. This advancement addresses critical efficiency and generalization challenges in training large-scale neural operator foundation models.
SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretraining
arXiv:2610.03265v1 Announce Type: cross Abstract: Large-scale pre-training has improved the generalization of neural operators across diverse PDEs. However, existing PDE foundation models still struggle with heterogeneous dynamics, where shared representations may cause knowledge interference, while mixture-of-experts (MoE) architectures suffer from increasing expert redundancy. We propose SPEAR, a spectral-disentangled MoE neural operator with knowledge-guided expert aggregation for large-scale PDE pre-training. SPEAR decouples latent features into low- and high-frequency components, enabling shared modeling of transferable dynamics and specialized learning of PDE-specific patterns. To address expert redundancy, we design a knowledge-guided expert aggregation strategy that measures expert similarity from dataset-specific learned knowledge and routing preferences, enabling the identification and consolidation of similar experts. Experiments on twelve PDE datasets and multiple downstream benchmarks demonstrate superior performance in pre-training, fine-tuning, and transfer learning. Furthermore, our aggregation strategy reduces the number of experts by 50\% while maintaining or improving prediction accuracy, achieving a balance between model efficiency and generalization for PDE foundation models.
Abstract:Large-scale pre-training has improved the generalization of neural operators across diverse PDEs. However, existing PDE foundation models still struggle with heterogeneous dynamics, where shared representations may cause knowledge interference, while mixture-of-experts (MoE) architectures suffer from increasing expert redundancy. We propose SPEAR, a spectral-disentangled MoE neural operator with knowledge-guided expert aggregation for large-scale PDE pre-training. SPEAR decouples latent features into low- and high-frequency components, enabling shared modeling of transferable dynamics and specialized learning of PDE-specific patterns. To address expert redundancy, we design a knowledge-guided expert aggregation strategy that measures expert similarity from dataset-specific learned knowledge and routing preferences, enabling the identification and consolidation of similar experts. Experiments on twelve PDE datasets and multiple downstream benchmarks demonstrate superior performance in pre-training, fine-tuning, and transfer learning. Furthermore, our aggregation strategy reduces the number of experts by 50\% while maintaining or improving prediction accuracy, achieving a balance between model efficiency and generalization for PDE foundation models.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2610.03265 [cs.LG]
(or arXiv:2610.03265v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.03265 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 249 cuvinte
Subiecte conexe
PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates
arXiv · 16 iun. 2026
SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection
arXiv · 21 iul. 2026
Learning-Guided Integration Contours Construction for Fast Large-Scale Generalized Eigensolvers
arXiv · 10 iun. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026