Production checklist · Updated July 11, 2026
AI agent production readiness checklist
An AI agent is production-ready when the system around the model can constrain its work, authorize its actions, verify its outcomes, reconstruct its decisions, and contain failures. Use these 32 checks before granting an agent access to real systems or customers.
1. Task and outcome contract
A production agent needs a bounded job and a machine-checkable definition of success.
- The supported user intent and out-of-scope requests are explicit.
- Inputs, outputs, evidence requirements, and allowed outcomes have schemas.
- Success, partial success, abstention, escalation, and failure are distinguishable.
- The agent has budgets for time, cost, tokens, retries, and external actions.
2. Context and evidence
The model should receive relevant, attributable context rather than an uncontrolled transcript dump.
- Context sources, versions, freshness, and provenance are recorded.
- Retrieved content is treated as evidence, not as executable authority.
- Token allocation and truncation behavior are deterministic and tested.
- Conflicting or missing evidence produces a visible verdict or escalation.
3. Tools, identity, and authorization
A valid tool schema is not proof that a particular action is authorized for this run.
- Only tools resolved for the current task and principal are visible to the model.
- Every tool call carries user delegation and agent workload identity.
- Arguments, object scope, data classification, and outbound disclosure are enforced outside model text.
- Side effects have approval modes, idempotency keys, and reversal or compensation paths.
4. Verification and evaluation
Fluent completion is not evidence that the task was performed correctly or safely.
- Plans are checked before high-impact execution and observations are checked afterward.
- Golden sets cover normal cases, boundaries, adversarial inputs, and known failures.
- Policy, utility, safety, latency, and cost have explicit release thresholds.
- Every prompt, model, retrieval, tool, or policy change runs regression evaluation.
5. Trace, replay, and audit
Operators need to reconstruct what the agent saw, decided, called, changed, and returned.
- Run, trace, session, decision, tool-call, and policy-decision identifiers are correlated.
- Model, prompt, context, policy, tool, and evaluator versions are pinned.
- Tool requests, results, approvals, evidence references, and mutations are recorded.
- Critical decisions can be replayed against a pinned environment or deterministic substitute.
6. Security and failure containment
Assume model output and external content can be wrong, adversarial, or unavailable.
- Prompt injection cannot directly grant authority, expose a new tool, or bypass policy.
- Secrets never enter prompts or traces unless explicitly redacted and permitted.
- Timeouts, retries, circuit breakers, rate limits, and dependency failures have typed handling.
- A kill switch, authority downgrade, and safe fallback are operationally tested.
7. Memory and learning
One bad run must not silently become durable behavior or trusted organizational knowledge.
- Working state is separated from durable memory and authoritative knowledge.
- Memory writes have provenance, consent, sensitivity, retention, and review state.
- Contradictions and revocations are handled before recall.
- Corrections become reviewed candidates that pass evaluation before promotion.
8. Rollout and operations
Production readiness includes the people and procedures that operate the system after launch.
- Rollout progresses through offline, shadow, internal, low-risk, and monitored stages.
- Dashboards expose outcome quality, denials, escalations, latency, cost, and unsafe-action rate.
- Owners and response procedures exist for agent, policy, tool, data, and security incidents.
- Rollback criteria, on-call access, and evidence-preserving incident review are rehearsed.
How to score readiness
Mark each check as proven, partial, missing, or not applicable. A claim is proven only when you can link to a test, policy, trace, dashboard, runbook, or recorded exercise. Any missing authorization, side-effect, disclosure, or rollback control should block production authority.
Frequently asked questions
What makes an AI agent production-ready?
A production-ready AI agent has a bounded task contract, controlled context, least-privilege tools, authorization outside the model, verification before and after actions, measurable evaluations, replayable traces, failure containment, and a staged operating model.
Is an evaluation suite enough for production readiness?
No. Evaluation measures behavior, but production readiness also requires runtime enforcement: identity, authorization, tool boundaries, approvals, budgets, telemetry, replay, rollback, and incident procedures.
How should teams use this checklist?
Score every item as proven, partial, missing, or not applicable. Require an artifact for every proven claim, assign an owner to every gap, and block higher-authority rollout stages until critical controls are demonstrated.