Who Actually Controls Your AI Agents?

I spent most of 2004 watching people guess about their internal control over financial reporting while pretending they knew what they were doing, and it was a circus of bad documentation and blind optimism. I see the same expression now as firms deploy AI agents.

A chatbot that summarizes a PDF isn't an agent that can change a record or run a transaction, and there is a huge gap there. Most people aren't marking that difference in their risk registers. That mistake will get expensive when auditors show up for year-end testing.

Why ignore the legal side? US Congress is already drafting federal laws to mandate security standards for these agents. These aren't suggestions. When the government builds security scaffolding, they don't care about your best efforts. They want a design that stops unauthorized action before it starts.

If you're implementing an agent today, you probably have a prompt and some API keys. That isn't a control; it's a configuration.

Actual control design is about boundaries, and where does your agent's authority end? If it can move money or change permissions, you need a human in the loop, and the real question is how you prove that person actually did their job.

Too many people settle for control theatre. That's just keeping a log of things that happened. Real control is a block that stops an action unless specific conditions are hit; if your only evidence for an auditor is some giant CSV file you hope they don't read, you've already lost.

What happens when these controls fail? Grindr paid £26 million to settle a UK lawsuit because they shared users' HIV status with third parties. This wasn't some glitch. It was a failure to control the flow of special category data, and think about an AI agent with write access and a mandate to optimize user experience. Would it decide that sharing sensitive data is the fastest way to hit its goal?

The cost of a breach isn't just the fine. Look at Comcast paying out $117.5 million for Xfinity customers. That's the price of failing to secure the perimeter.

I keep hearing this from the innovation types. They say they just trust the security certifications from their LLM provider. They look at a SOC 2 report and assume they're safe.

They aren't.

Your vendor's SOC 2 report covers their cloud (which is fine) but it doesn't cover how you actually set things up. The model provider isn't managing the permissions on your agent. If that agent hallucinates some command that wipes out a production table or leaks PII to a third party API, the vendor won't be the one answering to the regulator. You will.

Then there is the insurance side of things, and your cyber insurers aren't stupid. They noticed the 10.5 million dollar Lemonade settlement and the other spikes in payouts; what happens when they realize you gave autonomous agents access to core systems without a documented control framework? Either your premiums spike or your coverage for automated errors just vanishes.

You need to move from "prompt engineering" to "permission engineering."

Stop wondering if the agent is smart enough to follow your rules, and start asking who has the power to change those rules and how you'll spot an unauthorized edit. If you can't show me a change log for system prompts that links back to a formal approval process, you don't have a control. You just have hope.

And hope is not an acceptable evidence artifact.

Look at how identity propagates. When an agent works for a user, check if it uses a generic service account with god-mode privileges or the limited permissions of that specific person; using a service account creates a hole in your audit trail. You won't be able to tell if an action came from a real request or a prompt injection attack.

Find whoever is piloting these agents in your org, and ask them for the authorization matrix and the kill switch. If they stare at you blankly, start budgeting for remediation costs now.