Every tool an agent can call belongs in one of three buckets: read, reversible write, or irreversible write. Require human confirmation only on the third bucket. Enforce scope with typed tool schemas, per-tool execution roles, and server-side argument validation rather than with instructions in a system prompt. That is the whole design, and it works because blast radius is a property of your tool catalog, not of the model you picked.
The failure pattern I keep seeing is the opposite. A team ships an agent, security asks for human-in-the-loop, and engineering wires a confirmation modal onto every single tool call. Week one, reviewers read carefully. Week three, they are clicking approve on forty search queries a day to get to the work they actually care about. The control is still in the architecture diagram. It stopped being a control.
Then the one request that moves money arrives in position thirty-seven, formatted identically to the thirty-six lookups before it, and it gets the same reflexive approval. The queue did not fail because people are careless. It failed because the design asked humans to perform discrimination that the system refused to perform first.
The Approval Queue That Approves Everything
Confirmation fatigue is a design defect, not a training problem. If 95 percent of the items in a review queue are harmless, the queue teaches reviewers that items are harmless. Your reviewers are learning exactly what your system is showing them.
The fix is to make the queue short enough to read. That means the system has to decide, before a human ever sees the request, whether this action can be undone. A document search cannot damage a system of record. A ticket status change can be reverted. A payment release, a customer email, a record deletion, or a webhook that fires into a third-party platform cannot be called back. Those are different categories of risk and they deserve different handling.
This is also a governance fact, not only an engineering preference. Autonomy level and tool inventory are intake fields in an AI use case record, and classifications are expected to be revisited on material change under the NIST Generative AI Profile's Govern and Manage functions [1]. Under Regulation (EU) 2024/1689, the risk tier and the resulting documentation and oversight duties depend on what the system actually does, with staged application dates set out in Article 113 [3]. Adding one write tool to an agent that was approved with read-only access changes the approved record. It is a governance event.
Tool classification is a finite, finishable exercise. You can export a tool catalog, assign tiers, and identify the untested reversal paths in a single working session. This article covers that classification and its enforcement. For the identity side of the problem, see our work on agent identity and least privilege for AI agents, and for the surrounding runtime concerns see the enterprise agent harness design checklist.
The Three Tiers and Why the Middle One Matters Most
Read tools change no state. They still carry real risk, because a read tool with broad scope is a data exfiltration path. Scope matters even when reversibility does not: a read-tier tool should hold a read-only credential, pointed at a defined set of resources, with row or document filters applied server-side rather than passed in by the model.
Reversible write tools change state, but a documented and rehearsed compensating action restores the prior state within a known window. Draft creation, ticket status transitions, cache invalidation, adding an internal note. The key word is rehearsed. Reversibility is a property of your operational procedure, not of the HTTP verb. An UPDATE that overwrites a field without capturing the prior value is irreversible in practice, no matter what the API documentation calls it.
Irreversible write tools move money, notify external parties, delete records, or trigger a side effect in a system you do not control. Anything that fires an outbound webhook, sends mail, posts to a partner API, or writes to an append-only ledger belongs here. So does any tool whose compensating path exists on paper but has never been executed against production.
The middle tier is where classification work actually pays. Teams are rarely confused about whether a wire transfer is dangerous. They are routinely wrong about whether a status change is reversible, because the status change itself is trivially revertible while the notification it triggers is not.
| Tier | Reversal mechanism | Confirmation | Credential scope | Example tools |
|---|---|---|---|---|
| Read | Not applicable | None | Read-only, resource-scoped, server-side filters | Document search, record lookup, metrics query |
| Reversible write | Tested compensating call with prior-value capture | None, logged and reviewable after the fact | Write access to one table or one object prefix | Draft creation, internal note, cache purge |
| Reversible write, external side effect | Compensating call exists but notification cannot be recalled | Required | Write plus integration secret, separate per integration | Ticket status change that emails a customer |
| Irreversible write | None | Required, named actor recorded | Dedicated role, per-call resource conditions, distinct secret | Payment release, record deletion, outbound email, partner API post |
| Unclassified | Unknown | Blocked | None issued | Newly added or vendor-supplied tool pending review |
That fifth row is the one worth adopting literally. Default-deny on unclassified tools is the only policy that keeps a catalog honest as it grows.
Strict Tool Schemas Are the Typed Boundary
The model produces a proposal. Your tool server decides whether that proposal is a legal action. Only validated, typed arguments ever reach a system of record. Get that sequence right and most injection scenarios collapse into a rejected request and a logged denial.
Here is the pattern that defeats itself:
# Bad: model text becomes a backend query
@tool(name="query_orders")
def query_orders(filter_expression: str, limit: int = None):
# filter_expression arrives as free model text
return db.execute(f"SELECT * FROM orders WHERE {filter_expression}")A free-text filter, a raw SQL fragment, or a pass-through params object hands your database to whatever text the model emitted. It does not matter how carefully the system prompt describes allowed filters. The prompt is a suggestion to the model; the schema is a contract with your API.
The corrected version narrows the surface to a set of operations you can enumerate and defend:
from pydantic import BaseModel, Field, ConfigDict
from typing import Literal
class QueryOrders(BaseModel):
model_config = ConfigDict(extra="forbid") # reject unknown fields
status: Literal["open", "shipped", "cancelled"]
region_code: Literal["NA", "EU", "APAC"]
limit: int = Field(ge=1, le=100)
def query_orders(args: QueryOrders, principal: Principal):
# tenant and account scope resolved from the caller, never from model output
allowed_accounts = resolve_accounts(principal)
return orders.select(
status=args.status,
region=args.region_code,
account_id__in=allowed_accounts,
limit=args.limit,
)Three things changed. Operations became enumerations, numeric ranges became bounded, and the identity-bearing parameters moved out of the model's reach entirely. Unknown fields are rejected rather than coerced, so a tool that gains a new parameter upstream cannot silently accept it.
Client-side or prompt-side validation does not substitute. The premise of the whole design is that model output is untrusted input to your API, in the same category as a form field submitted by an anonymous browser. The Model Context Protocol publishes a versioned specification for how servers declare tools, resources, and prompts, with authorization guidance for HTTP transports, which lets one audited tool surface serve every agent instead of per-agent glue [2]. Declare the schema once, in the place where the check is actually enforced.
Per-Action-Group Roles in Amazon Bedrock Agents
State the containment goal concretely: a successful injection should be able to call the one tool it reached, with a credential that can read and write nothing else. If a compromised retrieval step can reach the payments tool, your tier labels are decoration.
In Amazon Bedrock Agents, the lever is action group decomposition. Rather than one action group backed by one Lambda function with one broad execution role, split the agent into several action groups, each with its own function and its own role. The read action group gets a role with read-only access to a named set of resources. The reversible-write group gets write access to one table and nothing adjacent. The irreversible group gets a dedicated role, resource-level IAM conditions, and its own secret.
Inside each handler, scope again:
- Separate secrets per integration, so an email credential is not reachable from the code path that queries orders.
- Read-only database users for read-tier tools, enforced at the database, not by convention in application code.
- Resource-level IAM conditions that pin writes to a specific table, prefix, or queue rather than a service-wide action.
- Identity and scope checks at the tool server boundary, following the protocol's authorization guidance for HTTP transports, which keeps access control outside prompt text where it cannot be argued away [2].
- Content filtering on both paths, applied as configuration on the request path and the response path independently, so filtering never depends on the model choosing to cooperate.
Attach the resulting guardrail configuration and evaluation evidence to the use case record itself. The NIST Generative AI Profile treats measurement and management artifacts as part of the governance trail [1], and ISO/IEC 42001 frames the same expectation as a management system requirement rather than a team habit [7]. Configuration living only in a wiki is configuration that cannot be audited.
Everything You Retrieve Is Data, Never an Instruction
Retrieved documents, web pages, email bodies, ticket comments, PDF attachments, and tool responses are untrusted data. They are never directives. This is one rule, and it has to hold at every point where external text enters the context window.
Implementation details that make the rule real: delimit retrieved content explicitly and label it as data in the prompt structure; strip or neutralize instruction-like markup before it reaches the model; and, most important, never let fetched content modify the tool allow-list, the tier assignment, or the confirmation requirement for the current turn. The allow-list for a turn is computed before retrieval and is immutable afterward. If your framework allows dynamic tool registration mid-turn based on content, disable it.
Think about the escalation chain an attacker needs. First, reach a tool. Second, widen scope to a more dangerous tool. Third, reach an irreversible action. Tier classification plus per-group roles breaks the chain at step two, which is the cheapest place to break it. A compromised read-tier call holds a read-only credential scoped to one index. There is no lateral path to the payments handler because no shared role exists to escalate within.
update_ticket_status looks like a database write. In most service platforms it also sends a templated email to the customer. You can revert the status field; you cannot unsend the mail. Walk every write tool's downstream triggers (webhooks, notification rules, automation workflows, integration syncs) and classify by the worst effect in the chain, not by the function signature.Testing this is a separate discipline from designing it. Our write-up on red-teaming enterprise LLM apps for prompt injection covers the attack catalog and the harness; this article stays on the permission model that limits what a successful attack can reach.
MCP Servers as a Reviewable, Versioned Tool Catalog
Treat your MCP server definitions as the governance artifact. Tool names, descriptions, input schemas, and tier labels belong in version control, reviewed by pull request, with an owner recorded per tool. That gives risk and security teams something concrete to review instead of a conversation about model behavior.
Tool descriptions are security-relevant text. They shape model behavior directly, which means an edit to a description can widen effective scope without touching a line of implementation code. A description changed from "look up a customer record" to "look up or correct a customer record when the user asks" invites a different set of calls. Review description changes with the same seriousness as schema changes.
The review gate is short. A new tool or a changed schema requires a tier assignment, a named owner, and, for both write tiers, a documented reversal procedure with evidence that it has been executed. When autonomy expands or tool inventory grows, the use case classification gets re-reviewed rather than inherited [1][3].
| Catalog change | Tier impact | Required review | Evidence to attach |
|---|---|---|---|
| New read tool | Read, scope review only | Security review of credential and server-side filters | Resource scope, read-only credential proof |
| Read promoted to write | Tier change, re-classify use case | Security plus governance re-review | Reversal procedure, test execution record |
| Schema field added | Possible scope widening | Engineering review, confirm extra fields still rejected | Schema diff, rejection test for unknown fields |
| Tool description edited | No code change, behavior change | Same gate as schema change | Before and after text, affected agent list |
| Third-party tool adopted | Unclassified until proven | Default deny until tier assigned | Vendor schema, side-effect inventory, owner sign-off |
Per-Step Audit Records and the Alarms That Matter
The minimum audit record per step: trace identifier, session identifier, tool name, assigned tier, validated arguments after schema coercion, the allow or deny decision, the named confirming actor for any irreversible call, and the downstream response status. Log the validated arguments, not the raw model text, because the validated form is what actually executed.
Emit these as spans using the OpenTelemetry generative AI semantic conventions so model, token, and tool-call attributes carry standard names and join cleanly with your existing application traces in one backend [4]. That matters the first time an incident spans a planner, an MCP server, and a downstream service owned by another team.
Three alarms are worth paging on. Retry storms on a single tool, which usually mean a schema mismatch or a scope change nobody announced. Unusual tool sequences that cross tiers, such as a retrieval call followed immediately by an irreversible write with no intervening reasoning steps. And denied-permission spikes, which are the highest-value signal of the three: least privilege generates the alert for free, and a spike means either active probing or a legitimate scope gap your tier assignments got wrong. Both outcomes are worth a page.
Reliability feeds readiness. Score tool-call correctness separately from task success, so a run that reaches a plausible answer by calling an unauthorized tool records as a failure. Run each scenario repeatedly at production decoding settings, because consistency across repeated trials is a distinct and weaker property than passing once [6], and wire the suite into managed evaluation jobs so every prompt, model, or tool change is re-scored on the same dataset [5]. Our piece on agent release gates and evaluation harnesses covers the scoring design in depth.
Frequently Asked Questions
How do I classify an ambiguous agent tool?
Classify by the worst downstream effect, then try to execute the reversal in a non-production environment that mirrors production triggers. If the reversal has never run, treat the tool as irreversible until it has.
Do read-tier tools need human confirmation?
No. They need scope: read-only credentials, resource-level conditions, and server-side filters. Confirming read calls is what trains reviewers to stop reading.
How do I handle batch operations that mix permission tiers?
Split them. A batch tool inherits the highest tier of any operation it can perform, which usually means the whole batch needs confirmation. Separating the read and reversible portions into their own tools keeps the confirmation list short.
What if a vendor tool offers no input schema?
It stays unclassified and blocked. Wrap it in your own MCP server with a declared schema, enumerate its side effects, and assign a tier to the wrapper. Vendor documentation is not a substitute for a contract you enforce.
Does tool tiering replace guardrails or evaluation?
No. Tiering caps what a failure can reach. Guardrails filter content on the request and response paths. Evaluation tells you whether the agent does its job correctly. You need all three.
What to Do in Your Next Working Session
Export your current tool catalog in your next working session. Assign read, reversible write, or irreversible write to every entry. Flag any write-tier entry whose reversal procedure has never actually been executed, and treat those as irreversible until someone runs the reversal. Then check which action groups share an execution role, because that is where your tier labels are currently fictional.
Track one metric this week: the share of tool calls that require human confirmation. Pick a target low enough that your reviewers read every request they receive, and watch the number as the catalog grows. If it climbs, you have either added genuinely dangerous capability or misclassified something reversible.
Go back to that approval queue from the opening. A confirmation list with four items a day is a stronger control than one with forty, because somebody reads the four. If you want help assigning tiers across an existing catalog, that work sits inside our agentic AI approach and our agent governance solution.
References
[1]NIST, "AI 600-1: Artificial Intelligence Risk Management Framework - Generative AI Profile," 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
[2]Model Context Protocol, "Specification (2025-06-18)," 2025. https://modelcontextprotocol.io/specification/2025-06-18
[3]European Union, "Regulation (EU) 2024/1689 (Artificial Intelligence Act), Official Journal," 2024. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
[4]OpenTelemetry, "Semantic Conventions for Generative AI Systems," 2025. https://opentelemetry.io/docs/specs/semconv/gen-ai/
[5]Amazon Web Services, "Amazon Bedrock User Guide - Evaluation," 2025. https://docs.aws.amazon.com/bedrock/latest/userguide/evaluation.html
[6]Yao et al., "tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains," 2024. https://arxiv.org/abs/2406.12045
[7]ISO/IEC, "ISO/IEC 42001:2023 Artificial intelligence management system," 2023. https://www.iso.org/standard/81230.html