How public sector organizations can govern AI at scale

Every government CIO has the same mandate right now: adopt AI, move faster, do more with less. Coding assistants are making development teams more productive. Agents leveraging large language models are opening up new ways to serve citizens. And the organizations that don't move on AI risk falling behind.

But public sector organizations operate in a different reality than commercial enterprises. Before a single AI model or agent gets called, someone has to answer a set of questions that most AI tools or inference providers weren't designed to address. It’s not that public sector organizations are slow to adopt technology, but the stakes are just different. These organizations handle citizen data, defense information, law enforcement records, and public funds. The governance requirements that come with that responsibility are not optional.

Until recently, meeting those requirements meant saying no to AI, or at least saying "not yet." The tools available didn't give governance, budget, and compliance teams what they needed to say yes.

That's starting to change.

Here's what it looks like on the ground. A digital services team at a public sector organization starts using third-party coding agents, such as Codex and Claude Code, to help developers work faster. Developers love them, but then come the governance questions: Who has access to which models? Why is AI spend climbing so fast?

The compliance team asks where the data is going. It turns out the third-party coding agent platform is routing requests to infrastructure in another country. For an organization handling sensitive operational data, that's a problem.

The budget office asks about costs. No one can say how much the tools are costing or which teams are driving the spend. Response times are slow and unnecessary tokens are being used because requests are traversing infrastructure that was never optimized for the use case.

The result is predictable: the compliance team can't sign off, the budget office can't plan for it, and the AI project stalls. Not because teams don’t need AI, but because no one can answer the basic governance questions.

This pattern repeats across agencies and geographies. The specific regulations vary, whether it's GDPR, FedRAMP, or national sovereignty frameworks, but the underlying dynamic is the same. Teams want to use AI. Governance teams want to say yes. But the tools in the middle aren't giving either side what they need.

The public sector organizations that are making real progress on AI share a common trait: they solved governance before they scaled adoption. They didn't bypass the governance questions. They found a way to answer them centrally, so every team didn't have to figure it out on their own.

That's the role Unity Gateway plays within Databricks.

No single model fits every mission, so agencies need choice.

A citizen services assistant answering permit and tax questions around the clock handles high volumes in many languages, so it needs a model that responds quickly and costs little per request.

A fraud unit tracing patterns across benefits claims needs a model that can reason through complex evidence, and it can only run where that data is allowed to go.

A records office extracting details from scanned legal filings needs a model that reads documents and images, not just text.

A critical agency application is built on an obsolete code base, so updating it requires a frontier model that excels at complex coding tasks.

Choose one model for all three, and the agency either overpays for routine questions or under-serves its most demanding work. Governance has to make room for the right model for each mission while keeping data, users, and budgets under control.

What Unity Gateway gives governance and budget teams is straightforward. The organization controls which AI models, coding agents, MCPs, and AI tools are available and to whom. Teams get day-zero access to the latest frontier and open-source models, while administrators decide which models to enable and who can use them. New models don't become available to every user by default.

For example, at a revenue agency, that means an investigations unit can use AI on taxpayer records while access stays limited to the investigators authorized to work those cases. Spending limits and hard caps can be set by user, team, or use case, and every request is tracked and attributable. Compliance policies, including content guardrails and access controls, apply at the platform level to every AI interaction, regardless of which team or application initiates it.

Beyond setting spending limits, Unity Gateway helps organizations reduce AI costs with Smart Routing, which directs tasks to more cost-effective models while maintaining quality. For an agency modernizing legacy systems on a fixed budget, that means routine coding tasks go to lower-cost models, stretching the budget further without slowing developers down.

For organizations with strict data residency requirements, governance also has to cover where AI requests are processed. A national health service applying AI to patient records, for example, has to be able to show regulators where that data goes. Databricks manages data residency for its hosted foundation models by geography, and account admins can require that these requests be processed within their workspace's geography. Compliance teams get a clear, documented answer on where those requests are processed.

And because Unity Gateway supports models from multiple providers, open-source models, and external endpoints, teams aren't locked into a single vendor. They can use the best model for each task, while the governance team retains full visibility.

When organizations move their AI workloads to Unity Gateway, the dynamic that was stalling adoption reverses. Developers get access to the models they want through Databricks' infrastructure without each team needing to self-govern. The governance team gets a single view of AI activity across models, agents, and tools, with unified traces that capture requests, token usage, costs, and tool activity.

For the benefits agency, that means leaders can see that AI tools for processing claims are being used heavily and at a reasonable cost, so they can expand those tools to more caseworkers and get benefits to people faster.

The governance requirements that were blocking AI adoption become the unified foundation that makes it possible. When the compliance team has confidence that AI usage is visible and managed, and the budget office can see exactly where spending is going, saying yes to AI gets much easier.

Governments around the world are investing in AI while tightening data protection and sovereignty requirements. From citizen services to national security, the agencies that move fastest will be the ones that can match each mission to the right model, with the right controls.

Unity Gateway is how Databricks helps public sector organizations get there. It meets agencies where they are, with existing data governance in place, and adds AI governance on top. No parallel system. No multi-year implementation. Just the controls that public sector leaders need to move forward.

Want to learn more?

Do we need new infrastructure?

No. Unity Gateway is built into Databricks, allowing organizations to centrally manage AI access, security, and costs without deploying or maintaining a separate AI gateway or AI governance solution.

Can we control which models our teams use?

Yes. While Unity Gateway provides day-zero access to the latest AI models, administrators can control which models are enabled and who can use them. Organizations can approve models and manage access by user, group, or team, so teams can adopt new AI capabilities without compromising governance.

Can we set spending limits?

Yes. Budgets and hard spending caps can be set by user, team, or use case. Every request is tracked.

Does this work with multiple model providers?

Yes. Unity Gateway supports models from multiple frontier model providers, open-source models, and external endpoints.

Can we keep AI processing within our geography?

Yes. Account admins can require that requests to Databricks-hosted foundation models be processed within their workspace's geography. If a model isn't available in that geography, admins can choose to allow processing elsewhere.