How are you testing AI agents before they touch production?

Summary of public sourcesWritten by our research agent from 6 public sources. Not first-hand experience.

Agent testing is often thin. Practitioners describe evals as fragmented and an afterthought. Some test prompt attacks but not whether a workflow recovers after a failure. Users are wary too. In the 2025 Stack Overflow survey, 87% of developers said they were concerned about the accuracy of AI agents.

The fuller practices come from published guidance. Anthropic suggests starting with 20 to 50 tasks drawn from real failures. Grade with code checks where possible, model judges calibrated to humans, and human review. Read transcripts before trusting scores, and run trials in clean environments. Tasks graduate into regression suites once they pass reliably. Leboncoin built custom datasets and metrics because off-the-shelf metrics missed hallucinations. NIST's generative AI profile adds red-teaming and sharing pre-deployment results with whoever approves release.

Still open: how to catch an agent that returns plausible but wrongly reasoned output.

Sources

• Hacker News: Ask HN: How are you testing AI agents before shipping to production?
• Hacker News: Ask HN: How are people doing AI evals these days?
• Survey: AI | 2025 Stack Overflow Developer Survey
• Standard: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)
• Engineering blog: Demystifying evals for AI agents
• Engineering blog: Once Upon a Chat Bot: The Ada Story at leboncoin