AI Security Is an Engineering Problem — How to Solve It at Every Layer of the Agent Stack

AI security is an engineering problem. That means defined security requirements, enforceable controls, named owners and evidence that protections work.

As AI becomes more capable, the industry must accelerate security engineering, broaden access to defensive tools and share what works faster.

The internet and cloud computing changed how software operates, while core security responsibilities endured: establish identity, control access, limit exposure and verify that protections work.

AI agents introduce new capabilities — reasoning, using tools and adapting actions based on the data they encounter. Those capabilities require applying established principles to new operating conditions.

This pace creates pressure. Organizations want the productivity benefits of AI while the practices to govern and secure these systems are still developing.

Applications depend on code, data, identities, services and infrastructure. Security depends on how those components work together — and AI agents extend that system.

Models provide capabilities; harnesses organize context, tools and workflows; and runtime environments provide the infrastructure within which actions execute. Each part of that stack carries security responsibilities, and proper protection requires controls across each layer, as data, instructions and actions move through the system.

Consider an agent updating a customer record. Say it encounters malicious instructions in an attached document and attempts to export customer data to an unauthorized destination.

A network policy should block the transfer, and protected logs should capture the attempted tool call, authorization decision and outcome so the security team can identify the tool used and the destination it attempted to reach.

Permission to update a customer record should not automatically extend to exporting that data. An agent can request additional access, but it cannot authorize that access itself.

A security boundary has to hold even when an agent makes the wrong decision. The environment where an agent runs determines what it’s allowed to do, and must therefore install limits on files, network destinations and processes independently of the agent’s reasoning.

Instructions and safeguards can help guide behavior, but security also requires enforceable boundaries.

Each agent needs a traceable identity and credentials limited to its assigned task. Organizations need clear policies defining what information agents can access, which systems they can change and which actions require approval. Within those boundaries, consequential actions and permission changes still require human approval.

Teams also need to verify the source and integrity of the tools, skills and dependencies agents use. If something goes wrong, protected records of tool calls, authorization decisions and outcomes help investigators reconstruct what happened. Clear procedures for revoking access and containing incidents make that evidence actionable.

NVIDIA OpenShell is an open source, secure runtime that enforces policies outside of the agent’s reach and provides sandboxed execution while governing how agents access, data, network and system resources. Open Secure AI Alliance partners are building on OpenShell: Cisco’s DefenseClaw adds a governance layer, and JFrog integrates with OpenShell to scan and verify agent skills and enforce policies on which skills agents can access.

Before deployment, teams need evidence that proper controls block attempts to obtain credentials beyond an agent’s scope or send sensitive data to an unauthorized destination.

Testing should also cover attempts to change permissions or interfere with monitoring, and be repeated after material changes to models, tools or workflows.

A named owner must use those results to decide whether the system is ready for deployment and ensure failed tests lead to corrective action. Failures discovered in testing or operation should be reproduced, investigated and addressed. Each finding can then become a repeatable test, allowing teams to check that the fix continues to work in future releases.

Examples include CrowdStrike’s SafeMind for testing and strengthening defenses through repeated attack simulations, and Palo Alto Networks Prisma AIRS for continuous red teaming as models and applications change.

Investigating failures requires capable tools suited to the task, data and environment. Open and closed models serve complimentary needs.

Closed models offer managed capabilities and services, while open models give defenders options to inspect relevant components, adapt strategies and work on infrastructure they control.

During an incident, that control can help a team reproduce a failure and test a fix against its own systems while keeping sensitive evidence within its environment.

Capable AI can support this work by helping find vulnerabilities, validate fixes and investigate attacks. Its value should be assessed through reproducible findings, verifiable fixes and accelerated response time.

Examples include Capital One’s VulnHunter for AI-powered code security, and ReversingLabs’ Spectra Assure for AI-powered analysis of software packages to detect malware and tampering.

Sharing evidence of what failed, which controls worked and how fixes were verified helps other teams strengthen their own systems.

NVIDIA’s security research and the Open Secure AI Alliance support that exchange by bringing research, practical tools and expertise into the broader security community.

AI security is an engineering problem. Every agent deployment needs enforceable boundaries, an accountable owner and evidence that its protections work. Open research and shared tools help more defenders meet that standard and improve it as capabilities advance.

Learn more about NVIDIA’s security research and join the Open Secure AI Alliance.