aiqbench analysis
AMD's release of Instella-MoE-16B-A3B offers a significant open-source resource for researchers aiming to replicate end-to-end Mixture-of-Experts training pipelines on AMD Instinct hardware. While the model weights are restricted to academic use under ResearchRAIL, the accompanying MIT-licensed codebase and data mixtures provide immediate value for studying expert routing mechanisms like FarSkip-Collective and Gated MLA in production environments. This release directly addresses infrastructure-specific optimization challenges by demonstrating how MoE architectures can be scaled efficiently on MI300X/MI325X GPUs with minimal memory overhead.
AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs
AMD released Instella-MoE-16B-A3B, a fully open Mixture-of-Experts language model trained from scratch on Instinct MI300X and MI325X GPUs. It holds 16B total parameters but activates only 2.8B per token, using Gated MLA and FarSkip-Collective. AMD published weights from every training stage, plus data mixtures, configs, and inference code. The post AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs appeared first on MarkTechPost.
AMD released Instella-MoE-16B-A3B, a fully open Mixture-of-Experts language model trained from scratch on Instinct MI300X and MI325X GPUs. The model holds 16B total parameters but activates only 2.8B per token. AMD is publishing weights from every training stage, along with data mixtures, training configs, and inference code. Two systems-level choices carry the release: Gated Multi-head Latent Attention and FarSkip-Collective connectivity. Is it deployable? Partly.…
Excerpt shown under the publisher's RSS terms · 64 words · full text and figures are on the publisher's site
Related coverage
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
AMD · Jul 24, 2026
Instella-MoE Technical Report
arXiv · Sep 2, 2026
DI Series: Serving Kimi-K3 MXFP4 Wide-EP Disaggregated on AMD Instinct MI300X and MI325X
AMD · Oct 8, 2026
NVIDIA AI Releases Nemotron 3 Ultra: An Open 550B Mixture-of-Experts Hybrid Mamba-Transformer for Long-Running Agents
MarkTechPost · Jun 4, 2026