analiza aiqbench
This announcement is about making Google's TPUs a more practical target for serving large AI models. Developers can run vLLM and SGLang through a native PyTorch backend instead of the existing JAX-based paths, which matters most to teams that already work in PyTorch and want to customize or debug their serving stack. The performance and cost claims are vendor-published benchmarks on Ironwood TPUs, so readers should check them against their own long-context and agentic workloads before drawing conclusions.
Announcing New vLLM and SGLang Inference Engines on TPU

De la Google Developers Blog
FragmentThis post announces native PyTorch backends for vLLM and SGLang powered by TorchTPU, enabling developers to run and optimize serving workloads using familiar PyTorch tools. Built in collaboration with open-source project maintainers, these engines include optimized TPU kernel implementations, production-ready features like KV cache management with TPU Sync, and disaggregated serving. Public benchmarks demonstrate state-of-the-art performance and industry-leading cost efficiency on Ironwood TPU across long-context and agentic workloads.
As part of our broader announcements for PyTorch today, we are excited to launch native PyTorch backends for vLLM (developed in partnership with Inferact) and SGLang (developed in partnership with RadixArk).
For years, TPUs have powered large-scale AI workloads with high performance and efficiency. With the rise of agentic and multimodal workloads, serving efficiently is more important than ever. Both engines already run on TPU through JAX-based backends, but we kept hearing from developers that they wanted to customize and debug their serving stack in PyTorch.
So we started in earnest on TorchTPU, which runs PyTorch natively on TPU. We then built new vLLM and SGLang engines on top of it.
Extras afișat conform termenilor RSS ai publicației · 111 cuvinte · textul integral și figurile sunt pe site-ul publicației
Alte relatări ale acestei știri
- Google Developers Blog · acest articol
- vLLM Blog · Introducing vLLM TorchTPU
Subiecte conexe

[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
Latent Space · 21 oct. 2026
AMD and Cerebras Announce Disaggregated AI Inference
Cerebras · 23 iul. 2026
What Type of Inference is Active Inference?
arXiv · 12 iun. 2026
What Type of Inference is Active Inference?
arXiv · 4 iun. 2026