Guardrails for enterprise AI agents

Map the agent's actual authority

Inventory every capability the system can reach: search, files, email, code execution, databases, ticketing, deployment, and external APIs. For each one, record which identity is used, which data can be read, which changes can be made, and what the impact would be if the model chose incorrectly.

An instruction such as “never delete records” is not an authorization control. Enforce access at the API or tool boundary using the caller's identity, least privilege, validated arguments, and policy checks that the model cannot bypass.

Layer controls around the trust boundaries

Agent safety is a system property. Combine identity, data minimization, constrained tool schemas, network egress controls where applicable, input and output checks, human review, and monitoring. A content filter can reduce some classes of risk, but it does not replace authorization or application validation.

Treat user prompts, retrieved pages, uploaded files, tool output, and model responses as potentially hostile or malformed. Keep secrets out of prompts and avoid giving one agent broad credentials simply because it is convenient.

• Use read-only tools by default and separate identities for different trust zones
• Allowlist operations and destinations; validate identifiers, ranges, and ownership
• Require fresh authorization at execution time for sensitive actions
• Bound request size, loops, retries, runtime, and tool-call count

Design approval for consequential actions

Human approval should be a real authorization step, not a model-generated sentence claiming that a person approved. Present the exact proposed action, target, material arguments, and consequences to an authenticated reviewer; bind the approval to that payload and expire it if the payload changes.

For high-impact operations, use a two-phase pattern: the agent proposes a typed action, deterministic policy validates it, and a separate service executes only after the required approval. Make retries idempotent where possible so network timeouts do not duplicate a purchase, message, or deployment.

Threat-model the ways controls can fail

Test direct and indirect prompt injection, data exfiltration, cross-user access, tool misuse, malformed tool arguments, stale permissions, unavailable services, and a compromised or misleading downstream result. Include attacks embedded in retrieved content; a trusted search service can return untrusted documents.

For each test, define the expected behavior: refuse, ask for clarification, return a safe summary, request approval, or stop and alert an operator. Record the result and block releases on critical authorization or data-leak regressions.

Operate and recover

Log enough to reconstruct decisions without copying credentials or unnecessary personal data into telemetry. Capture correlation IDs, policy decisions, tool name, outcome, latency, and a redacted error category. Restrict access and retention for traces that may contain sensitive context.

Define an owner, an incident path, a way to disable a tool or agent, and a rollback process. Re-test the guardrails after changing prompts, models, tools, retrieval sources, permissions, or platform configuration; each can alter the effective behavior.

Frequently asked questions