An AI Agent Governance Framework for a Small Company

An AI agent governance framework is the written set of decisions about who owns an agent, what it may do without asking, how it is stopped, what is recorded and what evidence lets it do more. At a small company it fits on one page per agent, signed by its owners.

I wrote this for a CTO with no compliance staff and an agent a week from shipping. Someone has asked who owns it when it breaks, or a customer questionnaire wants your AI governance policy. Below is the page, a filled example, and a table that maps each line to a public text.

Nothing here is legal advice. I quote public frameworks exactly, describe each one as it describes itself, and say where I could not establish that a text applies to you.

What is an AI agent governance framework at small-company size?

At small-company size, an AI agent governance framework is one page with five sections: owners, tool tiers, the stop, logging and retention, and promotion evidence. Every line points at a person, a file or a flag. A line that points at nothing is left visibly blank.

The book uses the word “governance” as a discipline in one place. Chapter 23 of AI Agents, Engineered (in the full book) adopts a phrase from the field, to treat the agent like a privileged employee, and unpacks it into a defined role, scoped access and an audit trail. The chapter closes that section with this: “The badge, the ledger, and the manager are the parts you must build.”

I read the page as those three parts written down: the badge is the tool register, the ledger is the log and the promotion evidence, and the manager is the set of names.

What does one page replace?

It replaces one of two things. The first is a long policy copied from a template, naming roles a 30-person company does not have. The second is no document, so the first incident is also the first time anyone asks who may stop the agent.

A software engineer at a small startup described the second situation in a February 2026 forum question. The poster was “drafting up security related company policies which up to this point we have never had,” at a company where “by default almost everyone is admin to everything” (frenchtoast8, Hacker News, 2026). When I read the thread it had no replies.

This AI agent governance framework has eighteen lines. Eleven rest on the book, by quotation or close paraphrase, and seven are this post’s additions. I mark each one, because the book names no roles, no pager, no retention period, no promotion threshold and no incident process.

Who owns the agent?

Three named people own it: a behavior owner, an access owner and a pager owner. The three roles are this post’s addition; the book supplies the principle underneath them.

The principle is the last sentence of Chapter 12, Oversight and Autonomy (in the full book): “What has not changed, at any position of the dial, is whose signature it is.” The page turns that signature into names.

1.1 Role and scope is the book’s line. Chapter 23 asks for “A defined role” and says the system the agent runs in “should make doing anything else impossible rather than merely unlikely”. On the page this is two sentences: what the agent does, and what it must be unable to do.

1.2 Behavior owner is mine. This person answers for the prompt, the model choice and its settings, and the evaluation suite. Chapter 20 gives the reason these travel together: the chapter treats code, prompt, model settings and tool schemas as one versioned artifact, so that “an incident names exactly one artifact”.

1.3 Access owner is mine. This person grants and revokes every tool and credential. The practice they enforce is Chapter 17’s: “grant the minimum access the task requires, and nothing on standing”.

1.4 Pager owner is mine, and the book is silent on it. I found no mention of a pager or an on-call rotation in the six chapters behind this post. Chapter 20’s warning that “nobody makes good judgment calls at 2 a.m.” is about leaning on that person. The pager owner therefore carries out decisions the page already made by daylight.

What changes when one person holds two lines?

The page still works, with three rules that are my own. First, write the doubling on the page so a reader sees it. Second, the stop keeps two named people whoever holds what.

Third, the person who signs a promotion (line 5.3) is someone other than the behavior owner. The behavior owner produces the evidence, and a second reader should judge it. If the same person holds behavior and access, the pager owner signs promotions and countersigns any new tier 3 or tier 4 grant.

One person can hold all three owner lines; lines 1.4, 3.2 and 5.3 still need a second name. Until one exists I would write “no promotions” on 5.3 and keep the agent at the tightest dial position, which is my opinion and goes beyond the book.

Which consequence tier does each tool sit in?

Each tool sits in exactly one of four consequence tiers, and the tier decides its gate. This section is the book’s almost line for line, and the tier names match the site’s classifier tool.

Chapter 12’s rule is to “classify everything the agent can do into consequence tiers—read-only, reversible, externally visible, irreversible—and attach the oversight to the tier”. The oversight per tier follows in the same paragraph. Read-only actions run. “Reversible actions run freely too, provided the system logs enough to undo and audit them.” Externally visible actions go to a review queue, and “Irreversible or high-stakes actions wait for a signature, every time.”

2.1 Tool register is the book’s line: every tool the agent can call, with one tier beside it. Tier by what the action costs when it is wrong; the agent’s stated confidence plays no part.

2.2 Gate per tier is the book’s line. An approval gate is a point where the run halts until a person approves, rejects or edits the action. The chapter’s placement rule is that “the gate must live in your code, never in the agent’s judgment”. The page records where that code is.

2.3 Credential per tool is the book’s line, from Chapter 17: “Give each tool its own scoped credential, never your master key”. The page records the scope of each credential and where it is stored.

Two neighboring posts carry the depth. The guide to placing approval gates by consequence covers the gate itself, the approval card and the timeouts. The post on the blast radius of AI agents gives a sheet that prices each grant’s worst case, to attach to the page.

What does the stop halt, and who may pull it?

The stop halts every running loop and agent at once, and two named people may pull it. The first half is the book’s, and the second is this post’s addition.

Chapter 13 defines the kill switch as “one obvious, fast, tested way to stop every running loop and agent at once” and adds the test: “Test it before the first unattended run, the way you test a fire alarm before the building fills.” The chapter’s warning is short: “A kill switch discovered broken during the fire is a design document for the next system.”

3.1 What the stop halts is the book’s line with one addition. I add a list of what “every running loop and agent” means in your system (runs, queues, schedulers, pending approvals) and a second list of what the stop cannot undo.

3.2 Who may pull it is mine: two names, so that one person’s holiday does not remove the stop.

In 2012 a trading firm’s automated order router, a system with no AI in it, malfunctioned for about 45 minutes. The regulator’s order found that “Knight did not have supervisory procedures concerning incident response” and that the firm “needed clear guidance for its technology personnel as to when to disconnect a malfunctioning system from the market” (SEC, Release No. 34-70694, 2013).

3.3 Last drill is the book’s test, with the record added by me: the date, who pulled it, and the measured time from decision to last side effect. Chapter 12 adds a design note worth one field: “interruption is only cheap if resuming is”. Write down the state a stopped run is left in.

3.4 Incident contact and write-up is mine. The book has no incident process. The page names who is told first, within what time, and where the write-up is filed, with the artifact version at the top.

What is logged, and for how long?

Every tool call is logged with enough context to reconstruct it, and the retention period is a number the team chooses and explains. What to log is the book’s; the period is the reader’s, because the book gives none.

4.1 Action record is the book’s line, assembled from three chapters. Chapter 17: “Log every tool call”; the reason given is that “the trail of what it touched is the difference between an incident report and a shrug”. Chapter 23 adds the versions: you must be able to say “which version of the prompt, which model, and which configuration produced a given decision, and what changed when.” Chapter 15, Observability and Debugging (in the full book) adds the field no generic convention records: “whose authority an action was taken on”.

A forum commenter asked the question that last field answers: “Who’s accountable when the action executed three hops away from the human?” (niyikiza, Hacker News, 2026).

4.2 Content capture is the book’s line. Rendered prompts and tool results hold user data. Chapter 15’s instruction is to “treat content capture as a deliberate, redactable, off-by-default data flow, with retention limits and access rules”. The page records whether content is captured, what is redacted, and who can read a trace.

4.3 Retention period and reason is mine. I suggest two clocks, one for the action record and a shorter one for content. The split is consistent with Chapter 15’s “retention limits” and goes beyond what the chapter says.

Is there a retention period in any public text?

One, among the texts read for this post, and its scope is narrow. Article 26(6) of the EU AI Act says deployers of high-risk AI systems shall keep the logs that system generates, to the extent the logs are under their control, “for a period appropriate to the intended purpose of the high-risk AI system, of at least six months, unless provided otherwise in applicable Union or national law, in particular in Union law on the protection of personal data.” Article 19(1) sets the same floor for providers (Regulation (EU) 2024/1689).

That is what the text says for high-risk systems. I did not establish that it applies to a small company’s internal agent, and I read the regulation as first published, without checking later amendments. Ask counsel before relying on it.

A team may still adopt six months as a reasoned default and write that reason on the page, as a design choice that carries no claim of compliance. I found no published incident in which missing agent logs were the stated cause, so I cite none; the argument for this section is the book’s, in Chapter 15: “what you fail to capture is precisely what you will not be able to debug”.

What evidence promotes a task toward more autonomy?

Boring evidence promotes it: clean runs, sensible escalations, evaluation scores and a measured error rate. Those kinds of evidence are the book’s; the thresholds and the signer are the reader’s to write.

The autonomy dial is the book’s term for “how much an agent may do between moments of your attention, set per task rather than built into the system”. It has four positions: every consequential action signed, standing permissions, plan-level approval, and monitored autonomy.

5.1 Dial position per task class is the book’s line. The starting position is given too: “Start every new task class tight.”

5.2 Evidence for one notch is the book’s in kind and mine in number. Chapter 12 lists runs completed cleanly, escalations that were sensible, evaluation scores with honest sample sizes, and a measured error rate on a held-out set. It gives no count, so any count on your page is yours.

5.3 Who signs a promotion is mine. The book says where the evidence lives: “The ledger lives with you, in permission files and eval dashboards, because the agent cannot keep it.” It names nobody to sign.

5.4 Re-earn and review triggers is the book’s line. Chapter 12 says “the ledger’s evidence expires with the model version it measured” and then: “Re-earn the dial settings after every upgrade.” Line 5.4 lists the changes that reset a promotion.

How a change reaches users is Chapter 20’s subject, the rollout ladder. The guide on how to shadow deploy an agent and then canary it covers those rungs, and the rollout plan generator drafts the plan for one change.

Which lines make up the page?

Eighteen lines make up the page, in five sections. Each is marked Book or Own (this post’s addition), with the chapter behind it. A blank on line 1.2, 1.3, 1.4, 2.2 or 3.3 should block shipping; that rule is mine.

What does the blank page look like?

The blank page is plain text, short enough to print on one sheet and to keep in the repository beside the agent. The line numbers and names match the checklist above and the framework map below.

AGENT GOVERNANCE PAGE            Agent: ____________   Artifact version: ________
Signed (date, each owner): ______________________________________________________

1. OWNERS
1.1 Role and scope: does ______________________; must be unable to _____________
1.2 Behavior owner (prompt, model settings, evals): _____________________________
1.3 Access owner (tools, credentials): __________________________________________
1.4 Pager owner (woken first; backup): __________________________________________
    One person holds two lines? Which: __________________________________________

2. TOOL TIERS
2.1 Tool register (tier 1 read-only, 2 reversible, 3 externally visible,
    4 irreversible/high-stakes):
    tool ____________  tier __   (one line per tool)
2.2 Gate per tier (where the code lives): tier 2 log ______ tier 3 ______ tier 4 ______
2.3 Credential per tool (scope; where stored): __________________________________
    Blast-radius sheet attached: yes / no

3. THE STOP
3.1 What the stop halts: ________________________________________________________
    What it cannot undo: ________________________________________________________
3.2 Who may pull it (two names): ________________________________________________
3.3 Last drill: date ________ pulled by ________ measured halt time ________
    State a stopped run is left in: _____________________________________________
3.4 Incident contact and write-up: tell ________ within ________; filed at ______

4. LOGGING AND RETENTION
4.1 Action record (per tool call: time, tool, arguments, result, on whose
    authority, artifact version); stored at: ____________________________________
4.2 Content capture: off / on; redacted: ____________; readable by: ______________
4.3 Retention period and reason: action record ______ because ___________________
                                 content ______ because _________________________

5. PROMOTION EVIDENCE
5.1 Dial position per task class: task class ____________ position _____________
5.2 Evidence for one notch (clean runs, sensible escalations, eval score,
    measured error rate): _______________________________________________________
5.3 Who signs a promotion (other than 1.2): _____________________________________
5.4 Re-earn and review triggers: model / prompt / tool change; incident;
    owner change. Next scheduled review: ________________________________________

What does a filled page look like?

Here is the AI agent governance framework filled once for an invented case. The company, its agent and every number in this section are illustrative. A 30-person software company runs an internal support agent that reads tickets, looks up orders, drafts and sends replies, and can issue refunds up to a cap.

Section 1. Role and scope: answers billing tickets; must be unable to change account ownership or refund above the cap. Behavior owner: the product engineer who built it. Access owner: the backend lead. Pager owner: the CTO, with the backend lead as backup.

Section 2. I produced the tiers with the consequence tier classifier and copied them, so the table and the tool agree. The agent is new, so the dial sits at the tightest position and tier 3 waits for a signature too.

One wrinkle: the classifier places any action that moves money in tier 4. Chapter 23’s own clerk is looser (“the refund clears below a threshold and queues above it”), and a footnote in Chapter 12 says the tier boundaries “are judgment calls”. I print the stricter answer for a new agent.

Section 3. The stop is one flag. It halts running tickets, the queue of pending sends, the hourly scheduler and every pending approval. It cannot recall a sent reply or reverse an issued refund. The CTO and the backend lead may pull it; the last drill measured 40 seconds, and a stopped ticket returns to the human queue with its draft attached.

Incident contact: tell the pager owner within 15 minutes. The behavior owner files the write-up in the repository, with the artifact version at the top.

Section 4. The action record covers all five tools and is kept 12 months, because this invented company accepts billing disputes for that long. Content (ticket text, drafts, replies) is kept 30 days with card numbers redacted, readable by the three owners. Both periods are illustrative.

Section 5. Both task classes, replies and refunds, start with every consequential action signed. Replies move one notch after 200 consecutive sends approved without an edit, a passing evaluation suite and no escalation judged wrong. The backend lead signs. Refunds keep their signature.

I checked the example against the page before publishing. Each of the five tools has one tier, one owner and one log rule, and the stop’s list covers the paths all five run through.

Does the page still fill for a read-only agent?

Yes. Take an internal reporting agent with read-only access to a warehouse. Every tool is tier 1, line 2.2 reads “no gated tools,” and line 2.3 still names a credential scoped to specific tables.

The stop still halts the scheduler, and the action record still captures each query with the person it ran for. Line 5.1 may start further along the dial, since the agent changes nothing. A read is still a leak if the process can send data out, which the blast-radius post covers.

Which public framework does each line answer to?

Each line maps to a clause in at least one public text, or to none. The table quotes each clause exactly from its primary source. Use the buttons to see one text at a time; the last option lists the lines with no clause.

Each text describes itself. NIST says its AI Risk Management Framework (January 2023) “is intended to be voluntary, rights-preserving, non-sector-specific, and use-case agnostic, providing flexibility to organizations of all sizes and in all sectors” (NIST AI 100-1). NIST AI 600-1 (July 2024) calls itself “a cross-sectoral profile of and companion resource for the AI Risk Management Framework” and numbers its items as suggested actions (NIST AI 600-1).

The EU AI Act is a regulation. Every article quoted below addresses high-risk AI systems, their providers or their deployers. OWASP’s LLM06 is an entry in a community list of risks, and it introduces the logging mitigation I quote with a caveat: it “will not prevent Excessive Agency, but can limit the level of damage caused” (OWASP, LLM06:2025).

Singapore’s Model AI Governance Framework for Agentic AI (version 1.5, May 2026) says it “is targeted at organisations looking to deploy agentic AI, whether by developing AI agents in-house or using third-party agentic solutions” (IMDA, 2026). I describe it only as a model framework; I did not establish whether any part of it binds anyone.

Two findings stand out. Only one of these texts states a retention period; it is a minimum, and it is scoped to high-risk systems. And no framework I read says what evidence should promote a task toward more autonomy, which leaves lines 5.2 and 5.3 resting on the book and on your own numbers.

One standard is missing from the table. ISO/IEC 42001 is the management-system standard for AI that customers sometimes name. Its text is sold and I have not read it, so I map none of its clauses.

What does the page answer for regulated buyers?

A one-page AI agent governance framework answers “who, what and where” for a regulated customer’s review. It leaves open whether a rule applies to you. That second question belongs to counsel.

When a questionnaire asks whether you follow a named framework, the truthful answer has two parts. Say which lines of your page correspond to which clause, using the table. Say that you hold no certification, unless you do.

For AI agents for regulated industries, sector rules may set their own record-keeping periods. Chapter 23 notes that in a regulated setting the audit trail “is a hard requirement with a precise shape”. The page gives the fields; the periods come from the rules that bind you. The procurement side of the same conversation, the questions to put to a vendor, is in the checklist on AI agent reliability for enterprise buyers.

When is the page reviewed?

The page is reviewed after every model, prompt or tool change, after any incident, and when an owner changes. Anchoring the review to changes is my choice. It follows from the book’s rule that evidence expires with the model version it measured.

A review of the page is short. Re-run the classifier if a tool was added, re-run the stop drill if the stop’s code or the scheduler changed, and reset any promotion whose evidence predates the change. Then the three owners sign again with the new artifact version at the top.

I would add one calendar date as a backstop, since a system can go months unchanged. The interval is yours.

Where does this AI agent governance framework fall short?

It falls short in four places. The page is an index to enforcement and enforces nothing by itself: a tier written on paper with no gate in code is a wish.

Liability is outside the page. The names on the page are the internal answer to “who owns this.” What a contract or a court says about responsibility is a separate matter.

One page covers one agent. A company with ten agents needs ten pages and one shared stop, and at some size a risk function will want more than this.

And it rests on a limited reading: five public texts, quoted only where I could match the passage in the primary source, and one standard left unread. A row marked “none” means none in what I read.

One page, three names

An AI agent governance framework earns its keep on the day someone asks who may stop the agent and the answer is already written down. Fill it before the agent ships, and re-sign it when the agent changes.

Chapter 12, “Oversight and Autonomy,” develops the tiers, the gates and the dial behind sections 2 and 5 (in the full book), and Chapter 13 builds the stop. The agent security and operations guide places this post among its neighbors, or you can see the formats.