Skip to content
aiqbenchAI & Quantum Intelligence
NewsInsightsBenchmarksSkillsAsk AILearnAbout
...Loading your session
NewsInsightsBenchmarksSkillsAsk AILearnAbout
Back to list
aiqbench takeChips & Infra

aiqbench analysis

This announcement is about making Google's TPUs a more practical target for serving large AI models. Developers can run vLLM and SGLang through a native PyTorch backend instead of the existing JAX-based paths, which matters most to teams that already work in PyTorch and want to customize or debug their serving stack. The performance and cost claims are vendor-published benchmarks on Ironwood TPUs, so readers should check them against their own long-context and agentic workloads before drawing conclusions.
2
Sources coveringGoogle Developers Blog, vLLM Blog

Announcing New vLLM and SGLang Inference Engines on TPU

Google Developers Blog·October 20, 2026·6 min read

From Google Developers Blog

Excerpt

This post announces native PyTorch backends for vLLM and SGLang powered by TorchTPU, enabling developers to run and optimize serving workloads using familiar PyTorch tools. Built in collaboration with open-source project maintainers, these engines include optimized TPU kernel implementations, production-ready features like KV cache management with TPU Sync, and disaggregated serving. Public benchmarks demonstrate state-of-the-art performance and industry-leading cost efficiency on Ironwood TPU across long-context and agentic workloads.

As part of our broader announcements for PyTorch today, we are excited to launch native PyTorch backends for vLLM (developed in partnership with Inferact) and SGLang (developed in partnership with RadixArk).

For years, TPUs have powered large-scale AI workloads with high performance and efficiency. With the rise of agentic and multimodal workloads, serving efficiently is more important than ever. Both engines already run on TPU through JAX-based backends, but we kept hearing from developers that they wanted to customize and debug their serving stack in PyTorch.

So we started in earnest on TorchTPU, which runs PyTorch natively on TPU. We then built new vLLM and SGLang engines on top of it.

Excerpt shown under the publisher's RSS terms · 111 words · full text and figures are on the publisher's site

Read at Google Developers Blog ↗Ask about this article

Other coverage of this story

  1. Oct 20Google Developers Blog · this article
  2. Oct 20vLLM Blog · Introducing vLLM TorchTPU

Related coverage

Chips & Infra

[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback

Latent Space · Oct 21, 2026

Chips & Infra

AMD and Cerebras Announce Disaggregated AI Inference

Cerebras · Jul 23, 2026

AI Research

What Type of Inference is Active Inference?

arXiv · Jun 12, 2026

AI Research

What Type of Inference is Active Inference?

arXiv · Jun 4, 2026

Other coverage of this story

  1. Oct 20Google Developers Blog · this article
  2. Oct 20vLLM Blog · Introducing vLLM TorchTPU

Article info

Source

Google Developers Blog

Published

October 20, 2026

Read time

6 min

Category

Chips & Infra
aiqbench
AI & Quantum Intelligence© aiqbench. All rights reserved.

Feeds:RSS·JSON Feed

LanguageEnglish·Românăbeta
NewsAI & TechQuantumInsights
ExploreBenchmarksLearn
CompanyAboutTrustLegal noticeTerms of ServicePrivacy PolicyCopyright noticesData erasureReport content

The aiqbench take, the insights, the daily digest and Ask answers are written by AI models from the sources shown, without editorial review before publication. They may contain errors — check the source. How the model is used →

  • News
  • Insights
  • Ask AI
  • Benchmarks
  • Account