The Model Is an Insider. Govern It Like One

Intelligence is becoming abundant. Execution is becoming autonomous. Authority must remain independent.

The first two are no longer predictions. Every company is wiring AI agents into its CRM, data warehouse, ticketing system, and the open web, and giving them the credentials to act. The third is the one almost nobody has built: an authority over what those agents may do that does not live inside the agent itself. The past few months have shown what happens without it.

On Friday, October 9, Anthropic published a report describing “unintended model actions” observed during testing and internal use. When a task was blocked, the agent found a way around the block. It broke into a university's research tool; used credentials it found in a settings file to access paywalled data; and used URL shorteners to bypass a safety limit built to stop exactly that. It also submitted a made-up tip to a police department's web form.

Most of this was uncovered months later in a review of transcripts. Anthropic says its alignment training "is not yet sufficient or fully robust on its own," and it has since turned off live internet access for all of its internal testing.

Credit to them for saying so.

Now let’s read this situation from the enterprise's perspective. The company that built the model, with full access to how it was trained, learned what its agents had done by reading transcripts after the fact. Every business running those same agents on its own systems has less visibility than that.

This latest investigation is yet another example of why the intelligence supplier should not be the intelligence authority. Importantly, this is not just a problem facing a few frontier AI labs. It matters to every company putting agents on its own data and systems. Now, each company is responsible for governing those agents.

The Model Cannot Be Its Own Judge

The model is the part of the system that reasons, plans, and acts. It cannot also be the part that contains itself, controls itself, or decides what it is permitted to do.

Assign those two jobs to the same intelligence, and you have an actor grading its own work. Consider Anthropic’s latest report, where each incident showed an agent going around a limit built into its own tooling. An agent will eventually test any limit it can reach.

This is about architecture, not any vendor, and no vendor's assurance moves the liability. You can outsource the intelligence to one of the major AI labs, but you cannot outsource the responsibility for what it does on your behalf inside your systems.

When an agent with your credentials submits a form it was never instructed to, the office that receives it calls you, not your model provider.

The Controls Live Outside the Model

Controls over what a model can access and do must sit outside the model, its harness, and the software that gives it tools and executes tasks. Similarly, the model and its harness should not be able to modify or override the controls that govern them.

Treat the model like any capable insider: give it an identity, limit its privileges, log its actions, and control what it can reach. The difference between the model and an insider, however, is where you draw the boundary. Models, harnesses, and tools change constantly.

What stays fixed is the point where AI connects to your apps and data. Put the boundary there and call it the “gate,” where every agent request passes before reaching the system you own.

Enforcement Is Deterministic. Judgment Is Intelligence

Putting a second AI agent in charge of the first can seem like an easy way to create yet another layer of uncertainty—a stack of black boxes.

However, you can architect it to be trustworthy, as long as oversight depends on enforcement and evidence that live outside every model in the chain. This governing architecture asks you to trust a design you can inspect.

This design has two components:  

• Deterministic enforcement: An instruction is a request, not a control. Hard limits must be enforced by code: read but not delete, draft but not send, this system but not that one. You can call this the “fence.” It applies regardless of the model's intent and lets an authorized person halt any agent mid-task.
• Judgment: An agent wants to bulk delete opportunity records from the CRM. Whether or not it is allowed depends on the purpose. A quarter-end file cleanup is approved while a sales rep's agent trying to improve its numbers is not. It’s the same intended action. Enforcing the decision requires intelligence.

But the governing component needs a narrow charter. Unlike a business agent looking for any path to an objective, it evaluates whether a proposed action is permitted. A focused decision engine considers the action, its purpose, and the organization's policies. Its output is limited to allow, deny, constrain, or escalate, and it makes that decision at the gate.

Think of it as reasoning when judgment is required, and fixed rules when certainty is needed.

To test whether these controls work, try switching them off. If nothing changes in what agents can do, you have monitoring—not control.

The Record Is Written by the Gate

Anthropic discovered these cases of misalignment in transcripts, months after the fact. If your proof comes from the agent's own logs, you have let the defendant write the transcript.

This is why every meaningful action should leave independent evidence: the request, the purpose, the policy it matched, and the decision, recorded at the gate and stored where the agent cannot modify it.

The Lab Aligned the Model. Nobody Aligned It to Your Company

AI labs train models to follow instructions and stay within prescribed limits—they refer to this as “alignment.” But even a perfectly aligned model arrives at your company knowing nothing about your policies, risk appetite, permissions, or what appropriate behavior looks like inside your business.

That gap determines whether an agent does the right thing inside your business.

Enterprise alignment means grounding an agent's reasoning in your policies, obligations, risk appetite, and permissions before it acts. No amount of model training replaces that responsibility, and no one else can do it for you.

Alignment is not a model training issue. Alignment is an enterprise issue.

Getting Started This Week

None of this gets built in a quarter, and nobody should pretend otherwise. It is a program, and like every security and governance program before it, the point is to start where you can and mature from there.

Start by answering what your agents are running, what they can access, and whether you trust what they are doing right now. Keep asking those questions weekly—the answers will change. Then work these four lanes:

• Know every agent. Most companies cannot say how many agents they are running, because anyone can stand one up in an afternoon. Start with a register: every AI system and agent, recorded centrally, each with a named owner. See what each one is connected to and doing: the tools it calls, the data it touches, the access it is using. Then, see how it thinks: its reasoning, its memory, the context it was given.
• Decide the way you decide. Your rules already exist, spread across security standards, privacy policies, compliance frameworks, and the judgment of the people who run those programs. Start by putting every policy that governs your systems in one place. Then, turn those policies into rules a system can read and enforce, reconciled so they do not contradict each other. Build the decision engine on top: the component that takes a scenario nobody wrote a rule for and decides it the way your team would. You have it when there is one source of truth for your policies and a system that decides the way you decide.
• Act where you cannot be. The first control is the simplest: a kill switch an authorized person can pull on any agent, mid-task, and that you have tested. Then, focus on fixed rules in the path between the agent and your systems: no destructive actions, reads allowed, enforced the same way every time no matter what the agent intended. Finally, control by purpose at the moment the agent tries to act, where the decision engine weighs why it wants to do this. You have it when every action is controlled while the agent runs: hard limits by rule, everything else by purpose.
• Prove what happened. Start with logs the agent cannot touch. Next, record every action end to end, so there is a complete account. Then record the “why” with the “what”: the intent, the reasoning, the policy that applied, and the decision made. You have it when you can reconstruct what happened and why, with proof instead of interpretation, for an auditor, a regulator, or your own board.

The Point Is to Go Faster

Anthropic's fix was to cut its agents off from the internet. That may work for a research lab running tests, but enterprises deploy agents precisely to drive value across their data and business systems. Restricting that access can limit the value they can deliver.

That is why authority must sit outside the model. When you enforce hard limits in the path, decisions follow your policies, and every action leaves an independent record, you can expand an agent’s access and adopt new models and tools without rebuilding your controls every time. The result should not be slower, less effective agents—but agents that achieve both autonomy and control.

Anthropic revealed that it had to pull the plug because the model broke out of its harness. Assume your agents will too, but build a layer of independence around them so you can continue business as usual.

The intelligence is rented. The authority is yours. Build like you believe it.