What’s new in AI infrastructure and orchestration in October

The Google Cloud AI Infrastructure team had another busy month in October, filled with new product launches and updates to power today’s AI and agentic workloads scalably and efficiently. We were also honored to share our progress with customers, partners, and the broader community at leading industry events, including NVIDIA GTC, PyTorch Conference, and AGNTCon + MCPCon. Let’s dive in! Product, technology, and tools updates New compute service: Agent Compute Engine is a purpose-built compute service providing high-performance, hardware-isolated microVM sandboxes alongside expanded agent governance capabilities like Agent Gateway and Agent Registry, all in the service of securing and scaling autonomous AI agents. Its primary benefits include rapid sub-second activation, extreme workload density through copy-on-write memory sharing, enterprise-grade security and identity management, and cross-cloud observability. AI Hypercomputer updates: At NVIDIA GTC Berlin 2026 we announced updates to our agentic infrastructure, including to A5X instances powered by the NVIDIA Vera Rubin NVL72 platform, Virgo Network scale-out fabric, and Managed Lustre storage. These co-engineered advancements provide enterprises with accelerated performance, enhanced sovereign AI controls, while advanced Confidential Computing capabilities help you train and deploy complex multi-step models securely and efficiently. GKE product update: CPU startup boost for GKE, in preview, uses Kubernetes In-place Pod Resize to dynamically elevate container CPU resources during initialization and scale them back down to steady-state levels without having to restart the containers. This feature removes startup throttling and accelerates cold starts by up to 2x for demanding workloads, helping to lower cloud infrastructure costs. GKE product update: The preview of the Wiz Runtime Sensor add-on for Google Kubernetes Engine (GKE) brings low-touch deployment, automated lifecycle management, enterprise credential security, and fleet-wide governance to Wiz's eBPF runtime threat detection for both Standard and Autopilot GKE clusters. Modernization tools: Together at last. Google Cloud Modernize brings Google Cloud’s migration and modernization tools, including Migration Center, Google Cloud VMware Engine, Google Cloud Mainframe Modernization and our new EKS-to-GKE Migration Agent, into a single portfolio. Enterprise teams now have purpose-built agentic capabilities across infrastructure assessment, platform modernization, and application modernization. Cloud Run product update: Temporal Serverless Workers on Cloud Run automatically scale worker pools based on Temporal task queue demands and scale down to zero when idle. This integration eliminates cluster management, lowers costs, using Temporal Worker Versioning to safely handle stateful, long-running workflows and AI agent workloads. New open-source product: llm-d 1.0 is an open-source, Kubernetes-native serving stack that turns a cluster of model servers into a coordinated serving pool, unifying distributed LLM inference across mixed Google Cloud TPU and GPU fleets. Co-founded by Google and partners, it provides production-ready benefits such as cache-aware routing, tiered KV cache offloading, and priority-based flow control to dramatically improve context reuse, reduce time-to-first-token latency, and efficiently scale multi-tenant agent workloads. New open-source product: Working with Meta, we open-sourced TorchTPU, a native PyTorch stack that integrates Google TPUs directly into the PyTorch ecosystem alongside GPUs and CPUs. Now developers can train, serve, and deploy models natively on TPUs using familiar frameworks like Hugging Face, vLLM, and TorchTitan. Google Distributed Cloud updates: Organizations that operate under strict sovereignty mandates would love to use AI to help them process data, but can’t send it to the cloud. The latest version of Google Distributed Cloud (GDC) air-gapped brings frontier AI models and multimodal enterprise search with Gemini into these isolated environments. In addition, GDC air-gapped supports NVIDIA HGX B300 systems based on the NVIDIA Blackwell Ultra architecture, and features enhanced networking and security capabilities.  Edge inference product: Google open-sourced ML Drift, a cross-platform on-device GPU compute engine built for LiteRT that abstracts hardware complexities across OpenGL ES, OpenCL, Metal, and WebGPU. The framework delivers peak performance, enhanced 5D tensor support, and memory efficiency for both classical machine learning workloads and advanced generative AI models at the edge. Open-weight model support: Reflection's Beam is an open-weight, 501-parameter model released under an Apache 2.0 license designed specifically for frontier software engineering and long-horizon agentic tasks. We announced day 0 support and ready-to-run deployment recipes for service Beam on AI Hypercomputer and NVIDIA AI Infrastructure. Practitioner guides and how-tos A

Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site