analiza aiqbench
This is a model-serving release: vLLM adds day-0 support for MiniMax-M3.1-Flash, with kernel-level optimizations for NVIDIA Blackwell, Rubin, and AMD MI350-series GPUs. Most reported gains (1.1-3.3x speedups, latency and throughput figures) come from vendor-side benchmarks on specific hardware and workloads, so they are most useful as relative comparisons rather than general claims. For practitioners, the main takeaways are the NVFP4 quantization path for KV cache and MoE weights and the support for RL fine-tuning on the quantized checkpoint, though the model is currently available only in the nightly build.
vLLM Delivers Day-0 Support for MiniMax-M3.1-Flash
Day-0 vLLM support for MiniMax-M3.1-Flash on NVIDIA Blackwell, NVIDIA Rubin, and AMD MI350-series GPUs, with NVFP4 MSA kernels, NVFP4 MegaMoE, DSpark speculative decoding, AgentX results, and day-0 RL.
MiniMax-M3.1-Flash is the latest model in the MiniMax-M3 family, with improvements across reasoning, coding, agentic, and visual tasks. It is also built for inference efficiency, with a native NVFP4 KV cache, NVFP4-quantized MoE weights, and a DSpark draft model for speculative decoding. vLLM supports MiniMax-M3.1-Flash on day 0 on NVIDIA Blackwell GPUs, on NVIDIA Rubin GPUs with dedicated Rubin kernels, and on AMD MI350-series GPUs, with strong performance on agentic workloads. MiniMax Sparse Attention (MSA) with an NVFP4 KV cache for both the indexer and the attention path, plus NVFP4 MoE weights and a DSpark drafter. Blackwell MSA kernels use the new CUDA 13.4 QMUL4 instruction to dequantize NVFP4 into FP8 inside the kernel: up to 1.7× faster indexer QK decode and 1.3× faster sparse attention decode. NVFP4 MegaMoE with MXFP8 shared-expert fusion and early TMEM release: 20% lower kernel latency in prefill and 7% higher prefill throughput. Indexer decode context parallelism (DCP): up to 3.3× faster indexer per sparse layer at 512K context on 4× GB300. QKV GEMM + norm + RoPE + KV-insert fusion: 1.1–1.6× faster prefill at 512+ tokens and about 19% lower P50 TTFT at TP4. FlashInfer PrimiTS backend for the DSpark drafter attention: 1.3–2× faster…
Extras afișat conform termenilor RSS ai publicației · 200 cuvinte · textul integral și figurile sunt pe site-ul publicației