Cost Per Successful Outcome: The AI Agent Metric That Decides Survival

The agent was accurate. Finance killed it anyway.

A mid-market SaaS company built a customer operations agent that scored green across twelve quality metrics—task completion, faithfulness, retrieval precision, tool accuracy, latency, and hallucination rate all within target bands. The agent was measurably more accurate than the human team it was designed to replace.

Then the CFO opened her laptop. The cost per resolved ticket, including failed attempts, was higher than paying humans to do the same work. Not marginally higher—the agent-assisted workflow was 10 percent more expensive than the fully-human process. The agent was shut down.

The problem wasn't the agent's performance. The problem was that the evaluation framework measured whether the agent was right, not whether it paid for itself.

Why it matters

As organizations move AI agents from pilot to production in 2025, economic viability—not technical accuracy—is becoming the primary filter for survival. Teams optimizing solely for quality metrics are discovering too late that accurate agents can still be economically underwater, and traditional cost-per-call dashboards hide the true unit economics until a finance review forces the conversation.

The arithmetic that kills agents

The SaaS company's agent attempted every incoming ticket at $3.40 per attempt in fully-loaded inference costs. It successfully resolved 71 percent of tickets; the other 29 percent escalated to humans at $4.20 each.

For 1,000 tickets: $3,400 in agent attempts plus $1,218 in human escalations totaled $4,618—versus $4,200 for humans alone. But the real problem emerged in a different calculation.

Cost per successful outcome—total agent spend divided by successful resolutions—came to $4.79 per resolved ticket. The human baseline was $4.20. Every successful agent resolution cost 59 cents more than a human would have charged, because the cost of failed attempts had to be carried by the successes.

The evaluation harness saw a 71 percent resolution rate at high quality and reported success. Finance saw unit economics above the human baseline and reported failure. Both were correct—they were measuring different things.

Why teams don't track it

Cost per successful outcome is genuinely difficult to compute. It requires attributing failed-attempt costs across successes, accounting for double-payment on escalations (paying both the agent to try and the human to finish), and establishing the business value of an outcome—which for many workflows is a modeling exercise rather than a simple lookup.

Most teams instrument cost per call, see a comfortable number, and never perform the division that reveals the real figure. The gap between cost-per-call and cost-per-success can exceed a factor of two, growing as resolution rates drop.

The survival threshold

Cost per successful outcome must sit below the business value of that outcome, with margin for operational overhead. This threshold moves against you at scale—unlike traditional software, agent unit costs don't fall with volume because each task is its own inference run.

The SaaS company eventually pulled two levers: they reduced cost per attempt from $3.40 to $2.30 through architectural optimization, and narrowed deployment to ticket types where the agent was already economically strong. Cost per successful outcome dropped to $3.10 against the $4.20 baseline—roughly 25 percent cheaper than humans on a narrower scope. The agent survived.

The metric cuts both ways

A legal operations team built an agent to review contracts for risk clauses at $18 per attempt—the most expensive per-call workload in the company's AI budget and flagged for cost-cutting. But the agent caught a genuine risk clause in one of every nine contracts. At $162 per caught clause against tens of thousands of dollars in downstream exposure per miss, it was actually the highest-return agent in production.

Cost per call said cut it. Cost per successful outcome said expand it. Only one view was connected to what the work was worth.

These findings were first detailed by an AI evaluation researcher writing on Towards Data Science, based on direct observation of production deployments and finance reviews.