aiqbench analysis
This announcement is about making Google's TPUs a more practical target for serving large AI models. Developers can run vLLM and SGLang through a native PyTorch backend instead of the existing JAX-based paths, which matters most to teams that already work in PyTorch and want to customize or debug their serving stack. The performance and cost claims are vendor-published benchmarks on Ironwood TPUs, so readers should check them against their own long-context and agentic workloads before drawing conclusions.
Announcing New vLLM and SGLang Inference Engines on TPU

From Google Developers Blog
ExcerptThis post announces native PyTorch backends for vLLM and SGLang powered by TorchTPU, enabling developers to run and optimize serving workloads using familiar PyTorch tools. Built in collaboration with open-source project maintainers, these engines include optimized TPU kernel implementations, production-ready features like KV cache management with TPU Sync, and disaggregated serving. Public benchmarks demonstrate state-of-the-art performance and industry-leading cost efficiency on Ironwood TPU across long-context and agentic workloads.
As part of our broader announcements for PyTorch today, we are excited to launch native PyTorch backends for vLLM (developed in partnership with Inferact) and SGLang (developed in partnership with RadixArk).
For years, TPUs have powered large-scale AI workloads with high performance and efficiency. With the rise of agentic and multimodal workloads, serving efficiently is more important than ever. Both engines already run on TPU through JAX-based backends, but we kept hearing from developers that they wanted to customize and debug their serving stack in PyTorch.
So we started in earnest on TorchTPU, which runs PyTorch natively on TPU. We then built new vLLM and SGLang engines on top of it.
Excerpt shown under the publisher's RSS terms · 111 words · full text and figures are on the publisher's site
Other coverage of this story
- Google Developers Blog · this article
- vLLM Blog · Introducing vLLM TorchTPU
Related coverage

[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
Latent Space · Oct 21, 2026
AMD and Cerebras Announce Disaggregated AI Inference
Cerebras · Jul 23, 2026
What Type of Inference is Active Inference?
arXiv · Jun 12, 2026
What Type of Inference is Active Inference?
arXiv · Jun 4, 2026