aiqbench takeMachine Learning

Moonshot AI's Kimi K3 represents a significant architectural shift in open-source development, introducing the world's first 2.8-trillion-parameter Mixture-of-Experts model with native vision capabilities and a one-million-token context window. The integration of Kimi Delta Attention and Attention Residuals aims to solve long-horizon reasoning challenges while maintaining efficiency through sparse expert activation strategies like Stable LatentMoE. Although performance still lags behind top proprietary models, this release establishes new benchmarks for open-model scale and architectural innovation in the current AI landscape.

Sources coveringMarkTechPost, Fortune

What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Moonshot AI Releases Kimi K3: A 2.8 Trillion Parameter Open MoE Model With Kimi Delta Attention and 1M Context

MarkTechPostJuly 16, 20264 min read

Moonshot AI released Kimi K3 on July 16, 2026. It is a 2.8-trillion-parameter open MoE model built on Kimi Delta Attention and Attention Residuals, activating 16 of 896 experts. The post Moonshot AI Releases Kimi K3: A 2.8 Trillion Parameter Open MoE Model With Kimi Delta Attention and 1M Context appeared first on MarkTechPost.

Moonshot AI just released Kimi K3. It is a 2.8-trillion-parameter model with native vision and a 1-million-token context window. Moonshot calls it the world’s first open 3T-class model. What is Kimi K3? Kimi K3 is a sparse Mixture-of-Experts (MoE) model built on two architectural updates. Those are Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). Both change how information flows across sequence length and model depth. K3 targets long-horizon…

Excerpt shown under the publisher's RSS terms · 69 words · full text and figures are on the publisher's site

Read at MarkTechPost ↗Ask about this article