Context Engineering Beats Bigger Models for AI Root Cause Analysis

The shift from model size to data preparation

A fundamental assumption about AI-assisted incident response is being challenged. Observability engineers increasingly believe that the quality of data fed to large language models matters more than the reasoning power of the models themselves when diagnosing production outages.

This shift has practical implications: teams building AI into their incident workflows may see better returns from investing in context preparation than from upgrading to more powerful models.

Why it matters

Most organizations adding LLMs to incident response face a choice between agent-based systems that let models investigate freely and deterministic pipelines that pre-correlate signals. This research suggests the deterministic approach—once seen as limiting—may actually be more reliable and cost-effective for production use, fundamentally changing how teams should architect AI observability tools.

Testing reasoning versus context quality

Observability vendor Coroot recently published research that separates two variables often conflated in AI root cause analysis: the model's reasoning ability and the quality of data it receives. Engineer Nikolay Sivko designed an experiment to isolate these factors, as first reported by InfoQ.

The test used a controlled scenario: a Chaos Mesh experiment that injected network delay between a catalogue service and its Postgres database, causing query slowdowns and front-end 502 errors. Sivko deliberately included misleading signals in the context, such as query timings inflated by network latency.

He ran the same 9,800-token prompt against eleven models, asking each to identify the root cause, explain the failure chain, and suggest an immediate fix. Closed frontier models—Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro—all correctly identified the chaos experiment and recommended deleting both it and its schedule. Most larger open-weight models performed comparably, though results varied among smaller self-hostable options.

The agent versus deterministic debate

Two architectural patterns dominate AI-assisted root cause analysis. Agent-based designs give models tools to fetch telemetry as they reason, allowing them to pull unexpected signals during novel incidents. Deterministic designs correlate data upfront and present the model with a single prepared context.

Companies like Dynatrace have adopted the deterministic approach, using topology-based causal analysis that traverses dependency maps rather than running open-ended agent loops. This choice reflects growing industry skepticism about agent reliability.

Practitioners from ZenML and Incident.io report that multi-agent LLM investigations prove difficult to debug in production, leaving no clean error traces—only unpredictable prompt interactions. Engineers discussing the tradeoff on Reddit describe abandoning fully agentic designs for mostly deterministic workflows with narrow LLM steps, citing better reliability and lower token costs.

Cost and engineering focus

Sivko noted that a single inference call costs only a few cents even on frontier models, because correlation work happens before the model is invoked. He concluded that the reasoning component of AI root cause analysis "is basically solved" and that the real challenge now lies in "preparing the right, compact context for the model before you call it."

This framing aligns with emerging guidance from Anthropic, LangChain, and observability vendor Mezmo, all emphasizing context engineering—curating the smallest set of high-signal, compact context—as a core discipline for reliable LLM-based systems.

The research and analysis were originally reported by InfoQ.