Deploy a Production-Ready NVIDIA AI-Q Blueprint on Oracle Cloud Infrastructure

AI agents have changed a lot in the last two years. The first could only answer one question at a time. Then came multi-turn chat, where the model could keep some context across a session. Today, we have long-horizon agents. Systems that plan many steps, split work between sub-agents, keep context across a long task, and run tools in a safe sandbox.

The NVIDIA AI-Q Blueprint is an open source reference for this kind of agent. It is built on LangChain Deep Agents and the NVIDIA NeMo Agent Toolkit. You can use it for quick cited answers, or for longer research reports with sources.

This post shows you how to deploy AI-Q 2.0 on Oracle Cloud Infrastructure (OCI) using Terraform to create the OCI resources and Helm to install the workloads on OKE. By the end, you will have a working AI-Q endpoint in your own OCI tenancy, and one command to take it all down when you are done.

Who this is for: Developers and platform engineers comfortable with Kubernetes, Terraform, and the shell, and who want to run AI-Q on OCI rather than on a laptop.

What you’ll learn: How AI-Q’s multi-agent architecture maps to OCI services, plus the exact commands to provision, deploy, and open the blueprint from start to finish.

More background on the multi-agent architecture (such as intent router, shallow research agent, deep agent, planning sub-agent, researcher sub-agent), is on the AI-Q product page and the NeMo Agent Toolkit docs.

API keys: NGC API key from build.nvidia.com, format nvapi-… used both as the NVIDIA inference key and to authenticate to the NGC container registry (nvcr.io). Tavily API key from tavily.com, format tvly-…

NGC API key from build.nvidia.com, format nvapi-… used both as the NVIDIA inference key and to authenticate to the NGC container registry (nvcr.io).

Tavily API key from tavily.com, format tvly-…

Local tools: terraform 1.5 or later, kubectl 1.28 or later, helm 3.x or later, the oci CLI set up with your API signing key

Some basic knowledge of Kubernetes, Helm charts, Terraform, and the shell. LangChain or NeMo Agent Toolkit experience is nice to have, but not required.

AI-Q uses a multi-agent design. An intent router reads each user query and sends it to the right workflow.

Figure 1. The AI-Q multi-agent architecture. The intent router sends queries either to the Shallow Research Agent (fast, bounded tool-augmented search) or to the Deep Agent (a Planning sub-agent and a Researcher sub-agent that share a Filesystem layer and run skills in sandboxes)

The blueprint is built to be extensible. Every layer (models, tools, RAG backends, sub-agents, evaluators) can be swapped through YAML config or through the NeMo Agent Toolkit plugin system. We will use that extensibility in Parts 2 and 3 of this series.

The deployment uses Terraform for the OCI resources and Helm for the Kubernetes workloads. This gives a clean split between infrastructure and application, and one terraform destroy is enough to remove everything later.

Figure 2. The AI-Q deployment on OCI. Terraform creates the VCN, OKE cluster, Load Balancer, and Vault. Helm installs the AI-Q backend, frontend, and PostgreSQL workloads on OKE.

Table 1. OCI resources created by the Terraform modules in deploy/terraform.

Backend (aiq-backend): A FastAPI-based agent server that runs the AI-Q workflow.

Frontend (aiq-frontend): A next.js web UI exposed over NodePort 30080.

PostgreSQL (aiq-postgres): An in-cluster database for the job store, checkpoints, and summaries.

Total time: around 20 to 25 minutes. The full reference is in aiq-2.0/README.md.

Step 1. Configure Terraform variables

At minimum, set these variables in terraform.tfvars:

user_ocid, fingerprint, private_key_path (same values as your ~/.oci/config)

db_admin_password, used to bootstrap the in-cluster PostgreSQL, stored in OCI Vault.

nvidia_api_key, your NVIDIA NGC key from build.nvidia.com. Used for inference and to pull container images from nvcr.io.

tavily_api_key, your Tavily key from tavily.com, for web search.

Step 2. Create the infrastructure

This takes about 10 to 15 minutes. Terraform creates the VCN, OKE cluster, Load Balancer, and the Vault with the NGC and Tavily API keys encrypted at rest.

Check: terraform output should show values for oke_cluster_id and lb_public_ip. If either is empty, run terraform apply again – the apply is safe to repeat.

Step 3. Install AI-Q from the NGC Helm chart

The chart and container images are published on NGC, so there’s nothing to build locally. We point kubectl at the new OKE cluster, create the secrets the chart consumes, then helm pull and helm install.

3a. Configure kubectl for the OKE cluster

3b. Export the API keys

Reuse the same NGC and Tavily keys you put in terraform.tfvars. The NGC key does double duty. It’s both the inference key and the nvcr.io pull credential.

3c. Create the namespace and secrets

3d. Pull and install the chart from NGC

The OCI overlay (values-oci-ngc.yaml) is intentionally tiny — it only pins the frontend service to NodePort 30080 (the port the OCI Load Balancer health-checks) and names the ngc-secret image-pull secret. Image repositories, the Postgres init SQL, and the dynamically provisioned 10 Gi Block Volume PVC all come from the chart’s own defaults.

Check: kubectl get pods -n ns-aiq should show aiq-backend, aiq-frontend, and aiq-postgres pods in Running state after 3 to 5 minutes.

Step 4. Open AI-Q

Open http://<lb_public_ip> in your browser. You should see the AI-Q frontend.

Try a simple question first, for example, “What is the NeMo Agent Toolkit?”, to confirm the routing works. Then try a deeper one, for example, “Compare the top three open-source deep-research agents by benchmark score and cost”, to see the deep agent in action.

terraform apply fails on OKE creation with a quota error. Check the service limits for your compartment for “Cluster count” and “Node count”, and ask for more quota if needed.

Pods stuck in ImagePullBackOff. Check that the image-pull secret was created (kubectl get secret -n ns-aiq) and that your NGC_API_KEY was correct when you ran the kubectl create secret docker-registry ngc-secret command in Step 3c. To rotate, delete the secret and re-create it, then kubectl rollout restart deployment -n ns-aiq aiq-backend aiq-frontend.

postgres pod stays in Pending for more than 2 minutes. The Block Volume PVC didn’t get dynamically provisioned. Run kubectl describe pvc -n ns-aiq. Typical causes are the OKE CSI driver not running, the default StorageClass missing, or insufficient Block Volume quota. Check the storage class with kubectl get sc and your compartment’s Block Volume service limit.

Load Balancer IP comes back as null. OCI can take a minute or two after Terraform to finish the LB. Run terraform refresh and then terraform output lb_public_ip again.

Frontend loads but queries return 500. Look at kubectl logs -n ns-aiq deploy/aiq-backend. The most common cause is a wrong or missing NVIDIA_API_KEY or TAVILY_API_KEY in the aiq-credentials secret you created in Step 3c.

You now have a working AI-Q 2.0 deployment on OCI, and one command (terraform destroy) to remove it cleanly when you are done. A few things to keep in mind as you go further:

Cost: The OKE node pool and the Load Balancer keep costing you while they run. Destroy the stack between experiments, or scale the node pool down to zero.

Secrets: Terraform stores the NGC and Tavily keys in OCI Vault at provision time (for audit and disaster recovery), but the running pods read them from the aiq-credentials Kubernetes secret you created in Step 3c. To rotate, delete and re-create that secret with the new values, then kubectl rollout restart deployment -n ns-aiq aiq-backend. Editing terraform.tfvars alone won’t reach the pods.

Extensibility: Everything you just deployed is driven by YAML and by the NeMo Agent Toolkit plugin system. Swapping an LLM, adding a sub-agent, or plugging a new RAG backend is a configuration change, not a rewrite.

Clone the AI-Q in OCI repo and share on the NVIDIA Developer Forum the solution you built and what problem you solved.