Reproducing AMD MLPerf Inference v6.1 Submission Results

This blog shows you how to reproduce AMD submission results for MLPerf Inference v6.1 on AMD Instinct MI355X, MI350X, and MI350P GPUs using self-contained Docker images, publicly available quantized model weights, and a step-by-step benchmark recipe.

Reproducing AMD MLPerf Inference v6.1 Submission Results#

September 17, 2026 by Meena Arunachalam, Miro Hodak, Uma Kannikanti, Poovaiah Palangappa, Yamini Preethi Kamisetty, Rebecca Lee, Neha Matthews, Rajesh Poornachandran, Karan Verma, Jiawei Chen, Huasha Zhao, Mikko Lauri, Jesus Carabano Bravo, Nico Holmberg, Eliot Li.

9 min read. | 2296 total words.

MLPerf Inference is the industry-standard benchmark suite governed by MLCommons, an open engineering consortium. It covers a representative set of AI workloads (recommendation, language modeling, computer vision) and measures system-level throughput and latency under production-realistic query patterns. The closed division — the division in which AMD participates — enforces strict accuracy constraints (99% or 99.9% of the FP32 baseline, depending on the model) and permits quantization, compiler optimization, and hardware-specific tuning, provided accuracy constraints are met. Workloads are tested across multiple scenarios: Offline (maximize throughput with unconstrained batch size), Server/Interactive (maintain tail latency SLO under a Poisson query arrival process) and SingleStream (minimize latency for processing 1 sample).

AMD MLPerf Inference v6.1 submission is our fifth consecutive round of MLPerf Inference participation. For technical details on the submission see our companion blog.

Before running any workload, confirm your system meets the following requirements.

One or more AMD Instinct MI355X, MI350X, or MI350P GPUs. Eight GPUs are required for all single-node submissions. The multi-node GPT-OSS 120B submission requires 72 GPUs across 9 nodes.

ECC must be enabled on all GPU devices. Verify with: rocm-smi --showmeminfo ecc Contact your system administrator if ECC is not enabled; MLCommons rules require it for datacenter submissions.

ECC must be enabled on all GPU devices. Verify with:

Contact your system administrator if ECC is not enabled; MLCommons rules require it for datacenter submissions.

OS: Ubuntu 22.04 LTS or later (recommended).

ROCm: [PLACEHOLDER: ROCm version, e.g., ROCm 6.4] — install via the ROCm Quick Start Guide.

Docker: Docker Engine with AMD GPU passthrough configured. Verify your installation: docker run --rm --device=/dev/kfd --device=/dev/dri rocm/rocm-terminal rocminfo | grep "gfx"

Docker: Docker Engine with AMD GPU passthrough configured. Verify your installation:

Hugging Face account: A free account at huggingface.co and a user access token are required to download the gated model weights for llama2-70b and gpt-oss-120b. Generate a token at https://huggingface.co/settings/tokens.

Total disk space required across all workloads is approximately 5GB. See each model section for per-workload estimates including Docker image, quantized weights, and dataset. Note that dlrm-v3 dominates the total at roughly 1.2 TB on its own, and also requires about 3 TB of host system memory.

Llama 2 70B is Meta’s 70-billion-parameter open-weight language model fine-tuned for dialogue and summarization tasks. It serves as one of the core LLM benchmarks in MLPerf Inference, evaluated on the CNN/DailyMail summarization dataset. AMD submission uses MXFP4 (weight-matrix MX floating-point 4-bit) quantization, which aggressively reduces memory bandwidth pressure on the MI355X’s high-bandwidth memory while maintaining accuracy above the 99% of FP32 baseline target. The standard scenario tests end-to-end summarization throughput and latency; the interactive variant adds per-token latency constraints representative of a live chat application.

Download the OpenOrca dataset following the instructions in the MLCommons inference repository.

The llama2-70b model on Hugging Face is gated. You must accept the model license at https://huggingface.co/meta-llama/Llama-2-70b-hf before the download will succeed with your token.

Note: The commands below are for MI355X. To execute on a different GPU, change the config file as appropriate.

Run the offline scenario accuracy test to generate the mlperf_log_accuracy.json file:

The mlperf_log_accuracy.json file is processed to verify the accuracy of the offline scenario:

Run the server scenario accuracy test to generate the mlperf_log_accuracy.json file:

The mlperf_log_accuracy.json is processed to verify the accuracy of the server scenario:

Run the interactive scenario accuracy test to generate the mlperf_log_accuracy.json file:

The mlperf_log_accuracy.json is processed to verify the accuracy of the interactive scenario:

The accuracy run must produce a ROUGE score at or above the threshold value specified in the MLCommons inference repository.

GPT-OSS 120B is an open-weight 120-billion-parameter Mixture-of-Experts (MoE) language model. MoE architectures activate only a subset of parameters per token, making them throughput-efficient for large-scale inference while maintaining high model capacity.

Note: The commands below are for MI355X. To execute on a different GPU, change the config file as appropriate.

Run the offline scenario accuracy test to generate the mlperf_log_accuracy.json file:

The mlperf_log_accuracy.json file is processed to verify the accuracy of the offline scenario:

Run the server scenario accuracy test to generate the mlperf_log_accuracy.json file:

The mlperf_log_accuracy.json file is processed to verify the accuracy of the server scenario:

The accuracy run must meet the 99% threshold relative to the reference score. The exact reference score is documented in the MLCommons inference repository under the gpt-oss model directory.

AMD submitted multi-node results for GPT-OSS 120B at cluster scale using AMD Instinct MI355X GPUs. The topology used was as follows:

Pull the Docker image containing the required code and scripts. For example, for GPT-OSS 120B:

To enable a distributed SUT with ZMQ, choose one node in the cluster as the Head node where the SUT client will run. Get the IP address of the head node using:

To run the benchmark under the server scenario, use the run_harness.sh script to start the SUT client. Make sure device_count is set to the number of all the healthy GPUs across all the nodes in the cluster.

Use the distributed_async_server.py script to start a worker on each node, including the Head node. Use the IP of the Head node for the Head node IP:

The steps for running the benchmark for the offline scenario are similar to the steps for running the benchmark for the server scenario.

The run_harness.sh script with config-name set to offline_mi355x is used to start the SUT client:

DeepSeek-R1 is a reasoning-focused large language model. AMD submission runs the quantized S3_sq_a05_v2 checkpoint on MI355X with the SGLang inference backend. Scenarios: Offline and Server.

You start in /lab-mlperf-inference/code, where main.py lives. The -v $(pwd)/model:/model and -v $(pwd)/data:/data mounts keep the downloaded model and dataset on the host so they persist across containers.

The DeepSeek-R1 model on Hugging Face is gated. You must have a valid Hugging Face access token. Generate one at https://huggingface.co/settings/tokens.

The configs read /data/deepseek-r1/mlperf_deepseek_r1_dataset_4388_fp8_eval.pkl, so make sure the eval file ends up there (move or symlink it if the downloader nests it under a subfolder).

Run from /lab-mlperf-inference/code. Every run must pass --backend sglang.

setup_deepseek_accuracy_env.sh is a one-time step. It installs the PRM800K (math) and LiveCodeBench graders the scorer needs. Run it before your first check_deepseek_accuracy_scores.sh, otherwise scoring aborts with a missing-module error.

DLRM-v3 (Deep Learning Recommendation Model, version 3) is the recommendation workload in MLPerf Inference, and v6.1 is AMD first submission of it. Unlike earlier DLRM generations, DLRM-v3 is built around a Hierarchical Sequential Transduction Unit (HSTU): it treats each user’s interaction history as a token sequence, runs stacked causal attention layers over it, and scores 2,048 candidate items per request. It therefore stresses two subsystems at once — a ~1 TB sparse embedding table that no single GPU can hold and must be row-sharded across all eight GPUs, and a long-sequence attention stack that dominates GPU compute time. AMD submission runs the dense HSTU path in FP8 (E4M3) with FP32 accumulation and keeps the embedding table at the reference precision, reading it directly over the AMD Infinity Fabric™ (xGMI) mesh. It uses full-causal attention with no sliding window, at batch 64 on MI355X and batch 32 on MI350X.

Storage: budget roughly 1.2 TB of free disk for this workload — a ~964 GB checkpoint, a ~140 GB preprocessed dataset, and the container image. The host also needs about 3 TB of system memory to stage the checkpoint.

DLRM-v3’s sparse embedding engine maps peer GPU memory over AMD Infinity Fabric, which depends on the host stack as well as the container. Use Ubuntu 24.04 with Linux kernel 6.8 or newer, amdgpu 6.16.6 or newer, and host ROCm 7.2 or newer. On older host stacks the identical container fails during embedding-engine initialization.

Unlike the other workloads in this blog, DLRM-v3 is driven by a standalone runner repository rather than a single pre-built image. The runner provisions the container, builds the two components that must be compiled for the gfx950 target, stages the model and dataset, and runs the performance, accuracy, and compliance tests. This is the procedure AMD used to produce the submission results.

DLRM-v3 uses a trained checkpoint and a preprocessed synthetic streaming dataset, both published on MLCommons storage. The dataset and checkpoint are not included in the image.

Checkpoint: download from https://inference.mlcommons-storage.org/metadata/dlrm-v3-checkpoint.uri (~964 GB, sharded)

Host requirements (qualified MI355X 8-GPU box — run_gold.sh checks most of these before launch):

Supported KMD: amdgpu driver version ≥ 6.16.13

All runs go through the run_image.sh launcher, which provisions the container with GPU device access and the capabilities the sparse embedding engine needs for peer memory over the fabric. The harness shards the embedding table across all eight GPUs on its own, so no GPU-count flag is required.

Every scenario needs a warm Triton autotuning cache. The first run after a fresh container start compiles kernels on the fly and can miss the latency target near the throughput knee; re-run once the cache is populated.

The 99th-percentile latency of about 60.43 ms is well under the 80 ms Server bound, so the result is VALID.

DLRM-v3 is scored by grouped AUC (GAUC), and a valid result requires GAUC at or above 99.9% of the reference. Run the accuracy test for each scenario; MLPerf requires it to use the same FP8 configuration as the performance runs, which the launcher does by default. The script generates mlperf_log_accuracy.json and then scores GAUC against the reference:

AMD submitted lifetime GAUC on MI355X is 0.78629 — a PASS with margin.

AMD submission reports TEST PASS with zero unmatched entries on both systems.

Wan 2.2-t2v (Wan2.2-T2V-A14B) is a 14-billion-parameter text-to-video generative model. The model uses a Mixture-of-Experts architecture with two experts activated sequentially during the denoising process: a High Noise Expert active in the early denoising stages and a Low Noise Expert that completes the process.

Step 1: Prepare the Docker Container - Wan 2.2-t2v#

Step 2: Download the Reference Model and Dataset - Wan 2.2-t2v#

Download the Wan 2.2 model from within the container:

Step 3: Run the Benchmark Harness - Wan 2.2-t2v#

Wan 2.2-t2v is evaluated under two scenarios: Offline (maximize throughput) and Single Stream (one request at a time, measuring end-to-end latency).

Switch to the /app/mlperf/mlperf_inference/text_to_video/wan-2.2-t2v-a14b/ folder inside the Docker container. Use run_scenarios.sh to run the accuracy, performance, compliance, and, optionally, VBench benchmarks for both Single Stream and Offline scenarios:

Results are written under runs/. If you used the bind-mount above, that tree appears on the host as ./mlperf_outputs/.

Llama 3.1 8B is Meta’s 8-billion-parameter instruction-tuned language model, evaluated on the CNN/DailyMail summarization dataset. AMD submission uses MXFP4 quantization with the vLLM inference backend. Scenarios: Offline, Server, and Interactive.

Step 1: Prepare the Docker Container - Llama 3.1 8B#

Step 2: Download the Reference Model and Dataset - Llama 3.1 8B#

The configs read /data/cnn_eval.json, so make sure the eval file ends up there (move or symlink it if the downloader nests it under a subfolder).

Step 3: Run the Benchmark Harness - Llama 3.1 8B#

Run from /lab-mlperf-inference/code. Every run must pass --backend vllm.

Note: The commands below are for MI355X. To execute on a different GPU, change the config file as appropriate.

Offline Scenario - Llama 3.1 8B#

Server Scenario - Llama 3.1 8B#

Interactive Scenario - Llama 3.1 8B#

The check_llama3_1_8b_accuracy_scores.sh step reads the dataset from /data/cnn_eval.json and the model from /model/ by default, and writes accuracy.txt (ROUGE scores) next to each mlperf_log_accuracy.json.

Expect a certain degree of variability in your runs, generally within 3%. If you see larger deviations, they may be due to:

AMD MLPerf Inference v6.1 submission demonstrates the breadth and maturity of the ROCm software stack for production AI inference workloads. Across recommendation systems (dlrm-v3), large dense language models (llama2-70b), MoE language models (gpt-oss-120b), and text-to-video generation (Wan 2.2-t2v), every workload follows the same three-step recipe: prepare the container, download the weights and dataset, and run the benchmark harness.

This is AMD fifth consecutive MLPerf Inference submission, each one deepening toolchain integration, expanding the workload coverage, and improving performance through ROCm-native optimizations. The Docker images package all dependencies — ROCm runtime, quantization kernels, inference server, and harness configuration — so any MI355X, MI350X, or MI350P system with ECC enabled and sufficient storage can reproduce the results directly.

Run the benchmarks on your own system and compare your numbers against the table in the Expected Results section above. If you encounter issues or have questions, engage with the community:

AMD ROCm blogs: https://rocm.blogs.amd.com

AMD Cautionary Statement - https://www.amd.com/en/legal/copyright.html

The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. THIS INFORMATION IS PROVIDED ‘AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies. © 2026 Advanced Micro Devices, Inc. All rights reserved

analiza aiqbench

Reproducing AMD MLPerf Inference v6.1 Submission Results

Subiecte conexe