analiza aiqbench
PreMoE introduces a training-free framework that proactively compiles sparse Mixture-of-Experts variants by leveraging router logits to estimate expert utility. This approach enables significant sparsity—up to 50%—across models ranging from 30B to 718B parameters with minimal performance loss, offering distinct trade-offs between domain-specific specialists and efficient generalists. The method addresses a key deployment challenge in MoE architectures by allowing specialization without retraining or full-capacity inference overhead.
PreMoE: Proactive Inference for Efficient Mixture-of-Experts
arXiv:2505.17639v4 Announce Type: replace Abstract: Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization. We introduce PreMoE, a training-free framework that proactively compiles sparse MoE variants for targeted deployment scenarios. At its core is Predicted Expert Utility (PEU), a robust metric for estimating expert importance from router logits through high-confidence threshold filtering and logit transformation, which together stabilize utility estimation under aggressive sparsity. Using PEU scores computed on a small calibration set, PreMoE produces domain-aware expert rankings that can be used to compile either domain-specific specialists or high-efficiency multi-domain generalists, without any retraining. Across MoE models ranging from 30B to 718B parameters, PreMoE achieves up to 50\% sparsity with nearly no performance loss. It further exposes a practical deployment trade-off: specialists maximize in-domain efficiency, while synthesized generalists retain broader cross-domain capability at the same sparsity budget. Code: https://github.com/JarvisPei/PreMoE
Abstract:Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization. We introduce PreMoE, a training-free framework that proactively compiles sparse MoE variants for targeted deployment scenarios. At its core is Predicted Expert Utility (PEU), a robust metric for estimating expert importance from router logits through high-confidence threshold filtering and logit transformation, which together stabilize utility estimation under aggressive sparsity. Using PEU scores computed on a small calibration set, PreMoE produces domain-aware expert rankings that can be used to compile either domain-specific specialists or high-efficiency multi-domain generalists, without any retraining. Across MoE models ranging from 30B to 718B parameters, PreMoE achieves up to 50\% sparsity with nearly no performance loss. It further exposes a practical deployment trade-off: specialists maximize in-domain efficiency, while synthesized generalists retain broader cross-domain capability at the same sparsity budget. Code: this https URL
Machine Learning (cs.LG)
arXiv:2505.17639 [cs.LG]
(or arXiv:2505.17639v4 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 156 cuvinte
Subiecte conexe
Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging
arXiv · 30 sept. 2026
A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference
arXiv · 16 iun. 2026
MoEless: Efficient MoE LLM Serving with Serverless Experts
arXiv · 1 oct. 2026
Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
arXiv · 7 sept. 2026