NVIDIA links TensorRT and Dynamo-Triton for faster AI on multiple GPUs

NVIDIA is streamlining generative AI workflows by enabling a single TensorRT network to execute across multiple GPUs, the company says. The company’s TensorRT 11.0 now supports multi-device inference using NCCL-backed distributed collectives, and NVIDIA Dynamo-Triton release 26.07 manages the complex setup for users. This integration allows organizations to reduce request latency by trading GPU resources. By handling GPU coordination internally, Dynamo-Triton accelerates review-and-refine cycles in latency-sensitive applications like video generation, demonstrated with the NVIDIA Cosmos 3 Nano model, where Dynamo-Triton serves the 44,160 video tokens across as many as eight NVIDIA GPUs. NVIDIA Cosmos 3 Nano video generation uses NVIDIA Dynamo-Triton, which serves the 44,160 video tokens across as many as eight NVIDIA GPUs. This integration addresses the increasing computational and…