Sari la conținut
aiqbenchInformații despre AI & calcul cuantic
ȘtiriAnalizeBenchmark-uriSkill-uriÎntreabă AIÎnvațăDespre
...Se încarcă sesiunea
ȘtiriAnalizeBenchmark-uriSkill-uriÎntreabă AIÎnvațăDespre

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

Înapoi la listă
perspectiva aiqbenchCipuri & infrastructură

analiza aiqbench

The NVIDIA Vera Rubin NVL72 demonstrates a significant efficiency leap in agentic inference, delivering over 10x more throughput per kilowatt than the GB300 NVL72 while maintaining strict interactivity requirements for AI agents. This benchmark highlights critical hardware and software optimizations needed to scale complex reasoning workloads without sacrificing responsiveness or model quality.
3
Surse care acoperăNVIDIA, CoreWeave, HPCwire

NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0

NVIDIA·16 noiembrie 2026·7 min de citit·Articolul și perspectiva sunt în engleză.

De la NVIDIA

Text integral

Agentic workloads are demanding more from inference systems. AI agents carry growing conversation histories across turns and generate a mix of reasoning, tool calls, and responses. Inference systems must support users at scale while remaining responsive.

MLPerf Endpoints v1.0 evaluates this balance between serving capacity and responsiveness using coding and enterprise workflows through API endpoints. During performance runs, generated tool calls and workflow intent are checked against reference values. Separate SWE-bench Verified evaluations assess end-to-end task-solving capability. These checks evaluate whether faster serving preserves model quality.

In MLPerf Endpoints v1.0, provisional NVIDIA Vera Rubin NVL72 results show over 10x the throughput per kilowatt of provisioned power of NVIDIA GB300 NVL72 at a shared minimum end-to-end interactivity of 50 tokens per second per user.1

This post presents the benchmark results and outlines the hardware and software optimizations used in the submissions.

1MLPerf® Endpoints v1.0, Kimi K3, Standardized division, Available category, published November 9, 2026. NVIDIA Vera Rubin NVL72, 32 accelerators, result 1.0.1.kimi-k3.agentic_combined.1p, run 9: concurrency = 1,024 queries, TTFT P90 = 4,673.9 ms, system TPS = 41,414.2 tokens/sec, E2E average interactivity = 50.4 tokens/sec per user, operating power = 100.89 kW. NVIDIA GB300 NVL72, 48 accelerators, result 1.0.0.kimi-k3.agentic_combined.1, run 6: concurrency = 48 queries, TTFT P90 = 2,305.1 ms, system TPS = 2,210.4 tokens/sec, E2E average interactivity = 56.2 tokens/sec per user, operating power = 60 kW.

Comparison based on throughput per kilowatt at a shared minimum E2E average interactivity of 50 tokens/sec per user. Retrieved from endpoints.mlcommons.org on November 9, 2026. GB300 results verified by MLCommons Association. For the provisional Vera Rubin results: pending MLCommons Association verification, results are subject to change pending peer review. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use is strictly prohibited. See www.mlcommons.org for more information.

The MLPerf Endpoints v1.0 dataset captures these workload characteristics: coding trajectories accumulate history over many turns, while enterprise workflows reuse shared instructions, tool definitions, and business rules. Figure 1 shows how context length grows across model responses during extended agentic sessions.

Figure 1. Context length grows across model responses in the MLPerf agentic dataset

The submissions span Kimi K3, Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash. Table 1 compares provisional Vera Rubin NVL72 results with verified NVIDIA GB300 NVL72 results for throughput per provisioned kilowatt at a shared minimum end-to-end (E2E) interactivity threshold and for E2E interactivity at matched query concurrency.

DeepSeek-V4.1-Flash

Table 1. Throughput per kilowatt and E2E interactivity ratios for Vera Rubin NVL72 compared with GB300 NVL72²² MLPerf® Endpoints v1.0, Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B, Standardized division, Available category. GB300 results published November 4, 2026; Vera Rubin results published November 9, 2026, for Kimi K3 and Qwen, and November 10, 2026, for DeepSeek. Retrieved from endpoints.mlcommons.org on November 10, 2026. Comparisons select each platform’s best TPS/kW meeting the stated minimum E2E average interactivity, or best E2E average interactivity at the stated matched concurrency.

Records list: result ID; run number; concurrency (queries); TTFT P90 (ms); System TPS (tokens/sec); E2E average interactivity (tokens/sec per user); operating-point power (kW).Kimi K3: Throughput/kW: see Footnote 1. Interactivity at matched concurrency 1,024 queries: Vera Rubin (1.0.1.kimi-k3.agentic_combined.1p; 9; 1,024; 4,673.9; 41,414.2; 50.4; 100.89); GB300 (1.0.0.kimi-k3.agentic_combined.1; 16; 1,024; 15,223.0; 13,905.8; 16.8; 60). DeepSeek-V4.1-Flash: Throughput/kW at minimum E2E average interactivity 280 tokens/sec per user: Vera Rubin (1.0.1.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.1p; 6; 1,408; 427; 272,711.7; 294.49; 75.67), 24 GPUs; GB300 (1.0.0.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.1; 1; 16; 112.8; 3,054.0; 288.65; 7.50), 4 GPUs.

Interactivity at matched concurrency 1,024 queries: Vera Rubin (1.0.1.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.1p; 5; 1,024; 505; 204,207.7; 307.36; 63.06), 20 GPUs; GB300 (1.0.0.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.1; 8; 1,024; 4,073.5; 49,701.8; 56.65; 7.50), 4 GPUs. Qwen3.6-35B-A3B: Throughput/kW at minimum E2E average interactivity 410 tokens/sec per user: Vera Rubin (1.0.1.qwen3_6-35b-a3b.agentic_coding.1p; 8; 320; 75.2; 47,785.5; 411.9; 12.61); GB300 (1.0.0.qwen3_6-35b-a3b.agentic_coding.1; 2; 36; 65.3; 5,960.4; 531.6; 7.50). Interactivity at matched concurrency 256 queries: Vera Rubin (1.0.1.qwen3_6-35b-a3b.agentic_coding.1p; 7; 256; 65.9; 41,142.6; 501.0; 12.61); GB300 (1.0.0.qwen3_6-35b-a3b.agentic_coding.1; 7; 256; 111.7; 23,467.8; 154.5; 7.50).

Ratios are rounded as shown. GB300 results verified by MLCommons Association. Vera Rubin’s results are provisional. Pending MLCommons Association verification, results are subject to change pending peer review. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use is strictly prohibited. See www.mlcommons.org for more information.

² MLPerf® Endpoints v1.0, Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B, Standardized division, Available category. GB300 results published November 4, 2026; Vera Rubin results published November 9, 2026, for Kimi K3 and Qwen, and November 10, 2026, for DeepSeek. Retrieved from endpoints.mlcommons.org on November 10, 2026. Comparisons select each platform’s best TPS/kW meeting the stated minimum E2E average interactivity, or best E2E average interactivity at the stated matched concurrency. Records list: result ID; run number; concurrency (queries); TTFT P90 (ms); System TPS (tokens/sec); E2E average interactivity (tokens/sec per user); operating-point power (kW).

Kimi K3: Throughput/kW: see Footnote 1. Interactivity at matched concurrency 1,024 queries: Vera Rubin (1.0.1.kimi-k3.agentic_combined.1p; 9; 1,024; 4,673.9; 41,414.2; 50.4; 100.89); GB300 (1.0.0.kimi-k3.agentic_combined.1; 16; 1,024; 15,223.0; 13,905.8; 16.8; 60).

Figure 2 shows the tradeoff between throughput per provisioned kilowatt and E2E interactivity across at least seven concurrency levels per curve. The curves show how much throughput each system delivers at a given responsiveness requirement. The step-function plot does not interpolate between measured points to keep comparisons grounded in demonstrated performance rather than estimates.

Figure 2. NVIDIA Vera Rubin NVL72 and GB300 NVL72 throughput per provisioned kilowatt versus E2E interactivity on Kimi K3 in MLPerf Endpoints v1.0

Higher throughput per kilowatt enables organizations to run more agents simultaneously within the same power budget while meeting the same responsiveness requirement. At low concurrency, faster responses reduce inference wait times for latency-sensitive agentic tasks.

NVIDIA Rubin GPUs combine HBM4 memory, NVIDIA Tensor Cores, and sixth-generation NVIDIA NVLink to support the demands of agentic inference. HBM4 enables rapid access to model weights and conversation state, Tensor Cores accelerate matrix operations, and NVLink supports communication across GPUs. Context-aware serving and model-specific kernels build on these capabilities to reuse prior work, independently optimize prefill and decode, and accelerate token generation—translating hardware advances into greater inference capacity and faster interactions.

Agentic workloads repeatedly reuse conversation history while adding new instructions and tool results. Efficient serving must preserve reusable state, distribute requests across workers, and accommodate the different execution requirements of prompt processing and token generation. Prefix caching, state reuse, and NVIDIA Dynamo routing help reduce redundant computation and balance serving load, while disaggregated serving supports independent optimization of prefill and decode.

Figure 3 summarizes how context-aware serving and model-specific GPU execution support agentic inference.

Figure 3. Context-aware serving and optimized GPU execution for agentic inference

Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B also have distinct architectures that benefit from targeted GPU execution improvements. Model-specific kernels, fusion, optimized memory access, and speculative verification reduce data movement and execution overhead. Table 2 summarizes how these serving and execution techniques work together across the three models.

Reuse previously processed conversation state and distribute requests to reduce recomputation and balance worker load.Example: Qwen’s fine-grained prefix matching reuses previously processed history across turns.

Configure prompt processing and token generation independently through disaggregated serving, reducing contention and synchronization waits.Example: Kimi combines disaggregated serving with prefill changes that reduce GPU synchronization waits.

Tailor kernels, fusion, and memory access to each architecture, reducing kernel launches and data movement.Example: DeepSeek’s MegaMoE kernel uses Rubin GPU locality and matrix-multiply instructions.

Propose multiple tokens and verify them together, reducing sequential decoding steps.Example: Qwen’s multi-token prediction pairs token proposals with Tensor Core verification.

Table 2. Software optimizations improve context reuse, serving efficiency, and GPU execution

Agentic inference performance depends on co-design across hardware, system architecture, and serving software. The provisional MLPerf Endpoints v1.0 results show how the Vera Rubin platform brings these capabilities together to address the demands of agentic workloads and create more capacity to serve users and scale applications.

Text integral sub licența deschisă a publicației · 1301 cuvinte

Citește pe NVIDIA ↗Întreabă despre acest articol

Alte relatări ale acestei știri

  1. 16 nov.NVIDIA · acest articol
  2. 16 nov.CoreWeave · CoreWeave’s Multi-Rack NVIDIA Vera Rubin MLPerf v6.1 Results
  3. 17 nov.HPCwire · ASUS Ramps Up NVIDIA Vera Rubin NVL72 Shipments, Showcases AI Factory Reference Architecture at SC26

Subiecte conexe

Cipuri & infrastructură

Inference Performance Index: changing the way we measure inference capacity

Epoch AI · 9 nov. 2026

Cipuri & infrastructură

Hardware-Rooted AI Security That Won’t Slow You Down

NVIDIA · 2 iul. 2026

Machine learning

Statistical Inference under Performativity: From Policy Feedback to Self-Evolving Systems

arXiv · 3 nov. 2026

Produse AI

DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models

arXiv · 4 iun. 2026

Alte relatări ale acestei știri

  1. 16 nov.NVIDIA · acest articol
  2. 16 nov.CoreWeave · CoreWeave’s Multi-Rack NVIDIA Vera Rubin MLPerf v6.1 Results
  3. 17 nov.HPCwire · ASUS Ramps Up NVIDIA Vera Rubin NVL72 Shipments, Showcases AI Factory Reference Architecture at SC26

Article info

Source

NVIDIA

Published

16 noiembrie 2026

Read time

7 min

Category

Cipuri & infrastructură
aiqbench
Informații despre AI & calcul cuantic© aiqbench. Toate drepturile rezervate.

Fluxuri:RSS·JSON Feed

LimbăEnglish·Românăbeta
ȘtiriAI & tehnologieCuanticAnalize
ExploreazăBenchmark-uriÎnvață
CompanieDespreÎncredereInformații legaleTermeni și condițiiPolitica de confidențialitateNotificări privind drepturile de autorȘtergerea datelorRaportează conținut

Perspectiva aiqbench, analizele, rezumatul zilnic și răspunsurile din Întreabă sunt scrise de modele AI din sursele afișate, fără revizuire editorială înainte de publicare. Pot conține erori — verifică sursa. Cum este folosit modelul →

  • Știri
  • Analize
  • Întreabă
  • Topuri
  • Cont