Trustworthy AI Requires More Than Safe Models. It Requires Boundaries on Agentic Authority

A follow-up perspective on building measurable trust as AI moves from recommending to acting

The AI trust equation has two sides: assurance and authority. Can we establish that an AI system is trustworthy? And even if we can, what exactly should that AI be allowed to do?

In my comments to Newsweek following the recent Government AI announcement, I argued that the opportunity is broader than creating another government organization. We have an opportunity to build an ecosystem that brings government, industry, researchers, standards bodies, independent testing organizations, and, ultimately, insurers together around measurable AI safety and security.

That kind of ecosystem becomes more important as AI moves from recommending and generating content to agentic systems that can act on our behalf.

From AI Risk Management to an AI Trust Ecosystem

We are not starting from zero. NIST has already established an important foundation through the AI Risk Management Framework (AI RMF), and it is working to align that framework with international standards and related approaches. But frameworks alone are not the end state.

The larger opportunity is to turn principles into measurable requirements, testing, validation, and eventually market incentives. That is how trust can become commercially valuable rather than simply a compliance obligation.

The Automotive Model: Trust Can Scale Commercially

The automotive industry provides a useful analogy. Vehicle safety did not become an international discipline because one government created a single regulator. Governments established baseline requirements, industry developed engineering standards, manufacturers designed against them, and independent organizations tested performance. Certification, procurement, insurers, and customers then created economic incentives for safer products.

Commercial scale then helped drive international harmonization. Manufacturers benefit when technical requirements, testing methods and approvals converge across markets rather than requiring entirely different products and certification processes in every country. UNECE vehicle frameworks show how harmonized technical requirements and reciprocal recognition can reduce duplication and support broader market access.

STANDARDS → MEASURABLE REQUIREMENTS → INDEPENDENT TESTING → VALIDATION → MARKET ACCEPTANCE → INTERNATIONAL HARMONIZATION → BROADER MARKET ACCESS

AI can follow a similar path. Developers can engineer against defined trust requirements. Independent organizations can test them. Governments and enterprises can use validated controls in procurement. Insurers can recognize demonstrably lower-risk architectures. That gives technology providers seeking broad market acceptance a commercial incentive to build to interoperable requirements.

The objective should not simply be to regulate AI. It should be to make demonstrable trust economically valuable and commercially scalable.

NIST Provides the Foundation, Agentic AI Adds the Next Question

NIST’s AI RMF provides an important foundation for trustworthy AI, while its standards work and crosswalks support international alignment. The progression is increasingly clear: risk management can inform standards; standards can support testing and evaluation; testing can support validation; and validation can become meaningful in procurement and commercial markets.

Agentic AI introduces another question. It is no longer enough to ask whether an AI system is trustworthy. We also need to determine what authority a particular AI agent should be allowed to exercise.

From AI Assurance to Agentic Authority

My colleague Emanuel Pirker raised an important extension to this argument: we need a technical framework for agentic authority. Online banking provides a simple analogy. I can authorize another person to access an account while limiting what they can see, what transactions they can initiate, how much they can transfer, and which actions require another approval.

Identity and authority are related, but they are not the same thing. AI agents should work the same way. An agent should present a verifiable non-human identity and operate within an explicitly delegated envelope defining what it may access, which actions it may perform, on whose behalf it may act, what quantitative or contextual limits apply, when human approval is required, and when authority must be suspended or revoked.

IDENTIFY → AUTHORIZE → OBSERVE → ENFORCE → CONTAIN → ESCALATE

This extends Zero Trust into an agentic world. We need to establish not only who or what the agent is, but what that agent is authorized to do at that moment and in that context, and then observe whether its actual behavior remains within those boundaries.

Our CEO, Klaus Oestermann, puts it plainly: „The next question for enterprise AI is the authority we give an agent at the moment it acts. That authority has to be defined, enforced, and revocable where the agent executes. Autonomy without enforceable authority is unmanaged risk.“

Trust Must Extend to the Point of Execution

Focusing AI security only on the model misses where risk becomes enterprise impact. The model may run in a hyperscale cloud, private data center, AI PC, or at the edge. The agent still must act somewhere: calling an API, accessing data, executing code, changing a configuration, initiating a transaction, or interacting with operational technology and critical infrastructure.

That is why future assurance should ask more than “Is this AI trustworthy?” It should also ask: “What is this AI authorized to do, and can those boundaries be technically enforced, continuously validated and independently audited?”

The Commercial Model Is the Opportunity

Government can establish baseline expectations. NIST and other standards organizations can translate them into measurable frameworks. Industry can engineer to them. Independent organizations can test and validate them. Enterprises and governments can use them in procurement. Insurers can recognize measurable risk reduction. International standards bodies can help harmonize requirements across markets.

That is how a safety requirement can become an engineering standard, an engineering standard a procurement expectation, a procurement expectation a commercial advantage, and a commercial advantage ultimately an international norm.

Safety and innovation do not have to work against each other. If trust becomes measurable, enforceable, and portable across markets, it can enable greater autonomy and broader adoption. The next step is to connect AI risk management with verifiable identity, delegated authority, runtime policy enforcement, containment and human control, and ultimately an internationally recognized testing and certification ecosystem.
Because the next generation of AI will not simply answer our questions. It will increasingly act on our behalf.

WHO IS THE AGENT? WHAT IS IT AUTHORIZED TO DO? CAN WE STOP IT WHEN IT CROSSES THAT BOUNDARY?

Sources

• NIST, AI Risk Management Framework (AI RMF). https://www.nist.gov/itl/ai-risk-management-framework
• NIST, Crosswalks to the NIST AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework/crosswalks-nist-artificial-intelligence-risk-management-framework
• NIST, AI Standards. https://www.nist.gov/artificial-intelligence/ai-standards
• UNECE, World Forum for Harmonization of Vehicle Regulations (WP.29). https://unece.org/transport/vehicle-regulations