Governance and the EU AI Act for AI Agents
Risk, permissions, and oversight, without strangling the thing you are trying to build.
Why this paper exists
When an agent acts on your behalf, its actions are your actions. That is the whole of it. A chat tool answers a question and the responsibility stays with the person who asked. An agent sends the email, moves the money, updates the record, and the organization owns the result, the same way it owns the result when an employee does those things. Governance for agents is not a compliance department's problem bolted on at the end. It starts the moment you give an agent a job.
This paper is deliberately principle-level. It is not a legal brief, and nothing here is legal advice. What it gives you is a way to think about agent governance that a leadership team can actually hold, and a sense of where the EU AI Act fits into that, so you treat the regulation as a forcing function rather than a fog.
At a glance
Governance for agents is not a new discipline you stand up from scratch. It is the five things you already use to manage an agent: a job, context, autonomy, tools, and oversight, written down, owned by someone, and open to inspection. Make those explicit and you have most of your governance. The EU AI Act, when you get to it, mostly asks you to show that you did exactly that. And the goal is never maximum control. It is control matched to risk, so the agent is safe without being useless.
Governance is the five things, made accountable
If you have read Managing AI Agents Like Teammates, you already have the scaffold. Governance is those same five elements, each one now treated as a control with an owner.
The job is your first and cheapest control: a written scope, including what the agent must never do. The context you give it is a data-governance decision, what it may use and what it may not touch. Autonomy is the master risk dial, the difference between an agent that acts and checks and one that proposes while a human commits. Tools are permissions: least privilege, granted on purpose, with a trail of what was used. And oversight is the standing job: a named owner, a set rhythm, and a log you can actually read.
Nothing in that list is exotic. It is the discipline of running the agent well, written down so someone is accountable for it.
Match the control to the risk
There are two ways to get this wrong, and they look like opposites. The first is to govern nothing: the agent acts freely, and a single wrong action scales at machine speed. The second is to govern everything: the agent is locked down so hard it cannot do the work you built it for, the strangling in the subtitle. Neither is governance. Both are a failure to think about risk.
The move is to tier your agents by stakes. An internal summarizer that drafts meeting notes and an agent that touches customers or money do not need the same leash. Decide, per agent, how reversible its actions are and how much a mistake would cost, then set its autonomy and its oversight to match. High stakes mean low autonomy and tight oversight, earned upward over time. Low stakes mean more freedom. That single judgment does most of the work.
Where the EU AI Act fits
You do not need to be a lawyer to lead here. You need to recognize the shape. At a principle level, the EU AI Act is risk-based: the higher the stakes of how AI is used, the more it asks of you. It expects transparency, so people should know when AI is acting on, or deciding about, them. And it treats the organization deploying the system as accountable, separately from whoever built the underlying model. You do not get to point at the model provider.
Read that way, the Act is less a rulebook to fear and more a forcing function that rewards the discipline above. If your five elements are explicit, owned, and logged, most of what the regulation asks for, you already have in hand. The organizations that struggle are the ones who governed nothing and now have to reconstruct it under deadline. (Specifics of the Act phase in over time and vary by use. Treat this as orientation, not legal advice, and get proper counsel for your jurisdiction and use case.)
Governance beyond compliance
The trap is to treat governance as a checkbox, something you pass once and file away. In two sessions I ran with Michael Herkommer of Riskovate, an AI governance and risk advisory I work with, one with the AI thematic group at Innovationsledarna and one at the AI Game Changers Club, that was exactly the posture he kept pushing against. Riskovate's premise is "bridging technology, finance, and risk for responsible AI": getting the CIO, CFO, and risk perspectives into one room so AI becomes governable, not just compliant. As they put it, the challenge is not understanding the rules. It is translating them into everyday governance, moving from regulatory compliance to board-level accountability.
The organizations that handle agents well take the same posture. They do not treat governance as the paperwork that proves they were careful after something went wrong. They treat it as the thing that earns them the confidence to give an agent more autonomy in the first place. Same discipline, opposite posture: not a brake you apply reluctantly, but the reason you can press the accelerator at all.
The agent register
If you do one concrete thing, make it this: a one-page agent register. One row per agent, with five columns: what it does, its risk tier, who owns it, what it is allowed to touch, and when it was last reviewed. That is it.
It sounds almost too simple, and that is the point. The register is the artifact that turns "we have some agents running around the business" into something a leadership team can actually govern. It surfaces the agent nobody owns, the one with broad permissions and no review, the one whose job was never written down. You cannot govern what you have not listed.
What this changes for the leadership team
Built in from the start, governance lets you move faster, not slower, because matched control and a clear owner are what let you grant more autonomy without flinching. Without it, leadership teams tend toward one of two bad equilibria. They freeze, banning agents until some imagined day when it is all safe, or they sleepwalk, letting agents act unwatched until a customer or a regulator finds the problem first. The register plus matched control is the path between those, and it is a posture, not a project.
What to do this week
Stand up the one-page agent register and fill it in for every agent you can find. Assign each one an owner and a risk tier. Then look for the single agent with the most autonomy and the least oversight. That gap is your first job, and closing it is worth more than any policy document you could write this quarter.
Frequently asked questions
Do we need to worry about the EU AI Act if we are not based in the EU? At a principle level, assume the discipline travels. If you serve EU customers or markets, parts of the Act can reach you regardless of where you are based, and even where it does not, the underlying expectations (risk-based control, transparency, a named accountable owner) are becoming the global baseline. Build for them once. For whether and how the Act specifically applies to you, get proper legal advice.
Isn't governance just going to slow us down? Only if you do it as blanket restriction. Done as matched control, light where the stakes are low and tight where they are high, governance is what lets you grant autonomy with confidence, which speeds you up. The slow path is the one where you have to rebuild trust after an unwatched agent does something expensive.
Who should own agent governance? Two answers. Each agent needs a named, accountable human, not "IT," a person. And the leadership team owns the register itself, because governance is a portfolio decision about risk, not a technical task to delegate downward.
How does this connect to the other papers? This is the accountability layer on top of Managing AI Agents Like Teammates, which covers the five elements themselves, and it assumes you have already chosen the right agents to build using Why / What / How. Govern the agents worth having, not a science fair of demos.