Direct answer
Seven tests before an AI agent belongs in production
An AI agent belongs in production only when its purpose, evidence, permissions, evaluation, observability, recovery, and accountable human owner are all explicit.
Purpose, evidence, and authority
The agent needs a bounded job, a definition of sufficient evidence, and a clear limit on what it may change. Broad prompts and broad credentials create an accountability gap even when the model performs well in testing.
The strongest designs separate detection, diagnosis, recommendation, approval, and execution. More autonomy is earned only where failure is reversible and the system can prove what happened.
Evaluation and observability
Final-answer quality is not enough. Teams need to inspect source selection, retrieval relevance, tool calls, latency, cost, rejected actions, and human overrides.
Evaluation should include normal work, ambiguous cases, stale knowledge, unavailable dependencies, and adversarial inputs. A system that only succeeds on the happy path is still a prototype.
Recovery is part of the product
Every operational agent needs a known response when a dependency fails: retry within a cap, use an approved fallback, queue the work, request human review, or stop safely.
Live canaries and rollback evidence matter because configuration, credentials, models, and external APIs drift. Production readiness is a continuing operating discipline, not a launch-day certificate.
Frequently asked questions
Questions leaders ask
What is the safest first use of an AI agent?
A narrow, observable workflow with read-only access or reversible actions, reliable evaluation data, and a named human owner.
Should agents receive administrator credentials?
Avoid broad administrator access. Use scoped, short-lived credentials and policy gates matched to the specific action.
How should an agent failure be handled?
Classify the failure, preserve evidence, stop repeated actions with a storm guard, follow a tested fallback or rollback, and escalate with decision-ready context.