Most organizations that adopted AI in the last two years have a governance document. Far fewer have governance. The document lives in a policy portal. The AI systems live in production. Nothing connects them, and everyone involved quietly knows it.

The failure starts with a mental model. Governance gets treated as a layer: something you add on top of a working system, the way you might add a WAF in front of a web app. Write the policy, hold the training, check the box. But a layer is static. It sits above the work and never touches it. The systems underneath keep changing, the layer does not, and within a quarter the two have nothing to do with each other.

Governance that works is a loop. Decisions produce policy. Policy produces scoped authority. Authority is enforced at runtime. Runtime produces evidence. Evidence feeds review. Review changes the decisions. Each stage exists to feed the next, and if any stage is missing, the whole thing degrades into theater.

Four things must stay in agreement: what a system can do, what it is allowed to do, what it actually did, and what you can prove it did. Governance is the discipline that keeps those four aligned. The moment any two drift apart, you have risk that nobody owns.

This matters more with AI than it did with traditional automation for one reason: AI-enabled agents and automated workflows can generate actions based on context you did not fully anticipate. A script does what it was written to do. An agent decides what to do next. Capability is no longer a fixed list you can review once. That makes the gap between capability and granted authority the single most important control surface in the architecture, and it makes runtime enforcement mandatory rather than optional.

Six stage AI governance loop: Decide, Codify, Scope, Enforce, Prove, Revisit

Figure 1: The governance loop. Each stage feeds the next. Break any link and the rest is decoration.

Stage 1: Decide

Everything starts with human judgment, and human judgment only counts when it is attached to a name. The first stage is deciding, explicitly, what AI is for in your environment, what risk you will accept, and who owns each decision. Not a committee. A person.

The core output of this stage is a set of decision rights: which classes of action an AI system may take autonomously, which require a human approval, and which are off the table entirely. Reading data versus writing data. Drafting versus sending. Recommending versus executing. Every environment draws these lines differently, but every environment has to draw them, because an undrawn line defaults to "the system does whatever it can," and that is the exact condition governance exists to prevent.

In practice

  • Build an accountability matrix. For every AI system in production, name the business owner, the technical owner, and the person who signs off on scope changes. If a system has no name beside it, it is ungoverned by definition.
  • Set autonomy tiers. Define three or four tiers, from read-only analysis up to autonomous execution, and assign every AI workload to one. The tier determines everything downstream: what gets codified, what authority gets granted, what enforcement applies.
  • Write down the refusals. The most valuable output of this stage is the list of things AI will not be permitted to do in your environment, regardless of capability. Production credential handling, unattended destructive operations, customer-facing commitments. The refusal list is short, explicit, and reviewed by leadership, not just engineering.

Stage 2: Codify

A decision that lives in someone's head, or in a meeting recording, is not policy. Codifying means writing the decisions down in a form that is versioned, reviewable, and unambiguous enough that an engineer can implement it without a follow-up meeting.

Treat policy like code because eventually it becomes code. Store it in a repository. Changes go through pull requests. History is preserved. When an auditor or an incident review asks "what was the policy on the day this happened," the answer is a commit hash, not an archaeology project through email threads.

In practice

  • One policy source, versioned in git. Markdown is fine. The format matters less than the discipline: a single authoritative location, change control, and named reviewers on every merge.
  • Write policy as testable statements. "AI systems should be used responsibly" is not implementable. "No AI service identity may hold write access to production identity stores" is. Every policy line should be checkable by a human or a script, and if it is not, rewrite it until it is.
  • Move enforcement candidates toward policy as code. Where the platform supports it, express rules in a policy engine such as Open Policy Agent so the written intent and the enforced rule are the same artifact. Not everything gets there. The gap between what is written and what is machine enforced should be known and shrinking, not unknown and growing.

Stage 3: Scope

This is where most real-world AI deployments fail today, and it is the stage that separates governance from paperwork. Scope is the deliberate act of granting an AI system less authority than it is capable of exercising.

A model wired into your environment through a service account with broad permissions has, in effect, been handed every capability that account carries. It does not matter what the policy document says. Authority is whatever the credential allows. If the credential allows more than the policy intends, the policy is fiction.

The common failure: an AI integration gets deployed under an existing service account "temporarily," the account has domain-wide or tenant-wide permissions accumulated over years, and nobody circles back. The system now holds authority nobody decided to grant, and Stage 1 never happened for it. This is the AI equivalent of running everything as domain admin, and it is exactly as defensible.

In practice

  • Dedicated identity per AI workload. Every agent, integration, and pipeline gets its own service identity. No shared credentials, no piggybacking on a human account, no reuse of the identity from the last project. If you cannot tell which AI system performed an action from the identity alone, scoping has already failed.
  • Least privilege, enumerated. Grant the specific API scopes, roles, and resource permissions the workload needs, and nothing else. Deny by default. If the platform offers granular scopes, use them all the way down. Broad roles exist for human convenience and have no place on a machine identity.
  • Time-box anything elevated. If a workload occasionally needs more, grant it just-in-time with an expiry, through the same approval flow a human would use. Standing elevated access on an AI identity is standing risk with no compensating judgment attached.
  • Inventory the credentials. Keep a live register of every AI service identity, what it can touch, and which policy line justifies each grant. This register is what Stage 6 recertifies against.

Stage 4: Enforce

Scoped authority defines the outer boundary. Enforcement is what happens inside it, while the system runs. This distinction matters because an AI system can do real damage without ever exceeding its granted permissions. An agent with legitimate write access to a ticketing system can still close a thousand tickets it should not have touched. The credential was valid. The action was wrong.

Runtime enforcement means controls that evaluate actions as they happen, not after. This is the stage the layer model skips entirely, and it is where the difference between a policy document and a governed system becomes physical.

In practice

1

Allowlist the action surface. Define the specific tools, endpoints, and operations each AI workload may invoke. Anything not on the list is blocked, logged, and alerted. Denylists do not work here because you cannot enumerate what a generative system might attempt.

2

Gate destructive and irreversible operations. Deletions, sends, payments, permission changes, and anything customer-visible pause for human approval. The gate lives in the execution path, not in the prompt. Instructions to a model are requests. Gates are controls.

3

Sandbox execution. Code execution, file manipulation, and network calls happen in isolated environments with explicit egress rules. The blast radius of a bad action should be decided by architecture before the action ever occurs.

4

Rate limit and budget. Cap actions per hour, spend per day, and records touched per operation. Runaway loops are a failure mode unique to autonomous systems, and limits convert a potential incident into a throttled anomaly.

5

Build the kill switch first. A single, tested, documented way to halt each AI workload immediately: revoke the identity, disable the integration, stop the runtime. If stopping the system requires a change ticket, you do not have a kill switch, you have a suggestion.

Stage 5: Prove

Every action an AI system takes either leaves evidence or it did not happen in any way you can defend. When something goes wrong, and eventually something will, the questions arrive in a fixed order: what did the system do, under what authority, who approved it, and what did the policy say at the time. If you cannot answer from records, you are answering from memory, and memory does not survive an incident review, an auditor, or a courtroom.

This is the stage that backup and recovery people recognize instantly, because the logic is identical to backup immutability. Evidence that the actor can alter is not evidence. If the AI workload, or anyone who compromises its identity, can modify or delete the logs of its own actions, the audit trail is worth exactly nothing on the day it matters most.

In practice

  • Log the full action record. For every action: the identity that performed it, the input that triggered it, the decision or output, the resources touched, any approval attached, and a timestamp. Model reasoning summaries are useful context, but the action record is the legal core.
  • Ship logs somewhere the actor cannot reach. Forward to a log platform or object storage under a different security domain, with write-only access from the source. Apply the same WORM and object lock discipline you would apply to backup data: retention set by policy, deletion impossible within the window, regardless of credentials.
  • Record approvals as first-class events. When a human approves a gated action, that approval is logged with the approver's identity and what exactly they saw when they approved it. "Someone clicked yes" is not an approval record.
  • Test retrieval before you need it. Run the drill: pick a past action and reconstruct the full chain from policy to permission to execution to evidence in under an hour. If the reconstruction takes days, the evidence exists but the capability does not.

Stage 6: Revisit

The loop closes here, and this is the stage that distinguishes a cycle from a straight line with good intentions. Everything upstream decays. Models get updated and behave differently. Permissions accumulate. New tools get wired in. Policy written for last year's deployment quietly stops describing this year's reality. Without a forcing function that feeds observed reality back into decisions, the loop breaks and you are back to a layer.

In practice

  • Recertify scope on a schedule. Quarterly, the named owner of each AI workload re-reviews its identity, its permissions, and its allowlist against what the workload actually did, using the Stage 5 evidence. Unused permissions get revoked. Unexplained actions get investigated.
  • Treat every gate trip and blocked action as signal. A spike in blocked actions means the system's behavior has drifted or the policy no longer fits the work. Either way, it goes to the Stage 1 owner as a decision, not to an engineer as noise.
  • Run incident findings through the whole loop. After any AI-related incident, the review asks which stage failed: was the decision wrong, the policy unclear, the scope too broad, the enforcement missing, or the evidence insufficient? The fix lands in that stage, and the change is versioned like everything else.
  • Re-baseline on model change. Swapping or upgrading the underlying model is a scope event, not a patch. Behavior changes, so the autonomy tier, allowlist, and gates get re-validated before the new model runs with the old authority.

Where the Frameworks Fit

None of this contradicts the published frameworks. It is what they look like when they touch the ground. The NIST AI Risk Management Framework organizes the same territory through its Govern, Map, Measure, and Manage functions. ISO/IEC 42001 wraps it in a certifiable management system with the documentation and continuous improvement machinery that auditors can assess. Regulatory regimes such as the EU AI Act add documentation and post-market monitoring obligations for systems in scope.

Use the frameworks for what they are good at: shared vocabulary, completeness checking, and external credibility. But do not mistake framework adoption for governance. You can hold a certificate and still have an agent running under a domain-wide credential with no runtime gates and mutable logs. The frameworks describe the loop. Only engineering closes it.

Closing

The test for whether you have governance or a governance document is simple. Pick one AI system in production and trace the chain: who decided what it may do, where that decision is written, which credential enforces the boundary, which control gates its riskiest action, where the evidence lands, and when someone last reviewed all of it. If the chain is intact, you have a loop. If it breaks anywhere, that break is where your next incident already lives.

Decisions without enforcement are language. Enforcement without decisions is machinery pointed nowhere. The loop is what connects intent at the top to execution at the bottom, and evidence is what lets you prove the connection held. Build all six stages, or accept that you built none of them.