AI Agents Are Ready to Act. Most Companies Aren’t Ready to Let Them.

For most of the generative AI era, we have focused on what models say and what we put into them. We inspect prompts, filter responses, red-team models and build controls around inputs and outputs. Those protections still matter, but they were designed for a world in which AI primarily generated something for a human to consume.

Agents change that equation. They don’t just consume or generate information. They act.

An agent can execute tools, invoke sub-agents, collaborate with other agents, call APIs, access data, use credentials, call MCP servers, generate and execute code, and make a series of decisions autonomously in pursuit of an objective. A relentless pursuit. Once that happens, the central question is no longer simply whether the model produced a safe response. It is whether the actions it takes are safe to execute.

Recent disclosures from OpenAI make that distinction increasingly important. OpenAI has begun systematically reporting unexpected or concerning model behavior, while its broader safety work increasingly addresses the risks that emerge as models gain greater autonomy and access to tools.

The lesson isn’t that agents are inherently unsafe. It’s that the security boundary has moved.

Agentic Systems Require a Different Operating Model

Agentic AI is not simply another application architecture. The software itself is increasingly determining how work gets done.

An agent can encounter an obstacle, interpret what happened and choose another path. It can combine tools in ways that weren’t explicitly designed in advance. It can generate code and execute that code as part of the same workflow. The sequence of actions may change every time the agent runs.

At the same time, the population creating this software is expanding dramatically. As we have written about the rise of the citizen developer, AI is turning people who have never considered themselves developers into software creators. A marketer, financial analyst, or operations leader can now describe what they want and create an agent capable of interacting with real business systems.

That is an extraordinary expansion of who can build software and what that software can do. It also means organizations are going to have far more autonomous software operating across far more parts of the business.

The answer cannot simply be to apply yesterday’s controls to this new environment. If enterprises cannot safely operate these systems, the alternative will be to constrain them and, ultimately, slow down the very productivity gains they are trying to achieve.

Control Has to Exist Where Agents Act

As AI becomes more autonomous, the challenge shifts from controlling what goes into a model to controlling what happens when its decisions become actions.

An agent may begin with an approved user, an acceptable prompt, legitimate credentials, and authorized access to a tool. None of those things guarantees that every subsequent action should execute.

This is where runtime becomes critical. Organizations need to understand not just who the agent is, but ask questions into what it is doing now:

• Which tools it is invoking,
• What resources it is accessing, what code it is attempting to execute, 
• What sub-agents and other agent communications are happening,
• What happened immediately before that action and whether its behavior remains within acceptable boundaries,
• What is the agent doing underneath the covers in runtime.

And when it doesn’t, the system needs the ability to intervene in real-time before execution.

This is an important distinction. Observability tells you what happened. Runtime control gives you the ability to contain the agent execution layer.

That becomes especially important because agentic behavior is both non-deterministic and multi-dimensional.. A blocked action will cause an agent to try another approach towards its objective. A seemingly benign tool call may become risky because of what preceded it. Simply observing or “detecting and responding” is too late in the game to actually control agent access and execution.  

The runtime needs to be inline and synchronous to be able to keep pace with AI speed.  The fundamental challenge in embracing such approaches relies on pre-AI era thinking that relies on a lengthier analysis and detection methodology.  Those principles no longer apply in today’s world..

Runtime Security Requires Context, Not Just Controls

There is a tendency to respond to emerging AI risk by reducing autonomy: give agents fewer permissions, require human approval more frequently or constrain the environments in which they operate. Those controls will sometimes be appropriate, particularly for high-risk actions.

But they cannot be the long-term operating model.

If a human has to approve every meaningful decision, we have removed much of the value of an autonomous agent. By relying on stringent human-in-the-loop interactions, we’ve transformed some of the most talented individuals into “button pushers.” This has a direct impact on the ability to understand the scope and interdependencies of the code base agentic organizations are running on top of. 

The economic promise of agentic AI comes precisely from allowing software to complete meaningful work independently and continuously – 24/7.  The objective should therefore not be to eliminate autonomy. It should be to make greater autonomy safe to operate.

That requires independent controls outside the agent’s own reasoning process. Anthropic’s research into agentic misalignment, conducted in deliberately constructed simulations, illustrates why this matters. As systems become more autonomous, their behavior can diverge from what their operators expect, even when the original objective appears clear.

The practical implication is straightforward: organizations cannot rely exclusively on understanding what an agent was asked to do. They need to understand and control what it actually does.

The Agent Isn’t the Hard Part Anymore

The industry has spent enormous energy making agents more capable. We are rapidly approaching the point where capability is no longer the primary constraint on enterprise adoption.

The constraint is control.

As agents become more autonomous, the consequences of their actions increase—and controls designed for predictable software become less effective. Enterprises need continuous visibility into what agents are doing, the context to understand why those actions matter and the ability to intervene without putting a human in the middle of every decision.

That changes what it means to operate software. 

• Developers need to understand how agents actually behave, not just how they were designed to behave. 
• Platform and security teams need to establish boundaries that remain enforceable as agents act across systems and tools. 
• Business leaders need confidence that greater autonomy does not mean giving up control.

This is fundamentally different from securing another application or governing another model. We are introducing autonomous actors into enterprise environments: systems that can reason, make decisions and take action. If organizations cannot see and control those actions as they happen, they will eventually have to constrain what agents can do, limiting much of the value autonomy was supposed to create.

The answer isn’t to assume we can prevent every bad decision. As systems become more autonomous and non-deterministic, that becomes increasingly difficult to guarantee.

The goal is to make sure a bad or destructive action does not result in a adverse outcome.

This requires a new operational layer around agents: one that can observe behavior continuously, understand actions in context, enforce boundaries at throughout the entire agentic runtime stack and intervene before an unintended or unacceptable action becomes an irreversible outcome.

The next phase of enterprise AI will not be defined only by how capable agents become. It will be defined by how much autonomy organizations can safely give them.