Key takeaways
- AI agent observability is the discipline of instrumenting, tracing and evaluating agentic systems in production. In 2026 it is a specialty of its own, distinct from traditional APM and from logging single LLM calls.
- Agents fail in multi-turn, multi-tool sequences: the wrong answer at step 10 usually originates in a tool call at step 3 or a retrieval at step 1. Without a full trace, diagnosis is guesswork.
- The rule that organises everything: evals tell you if the agent is good; monitoring tells you what it costs. You need both, and they measure different things.
- Without observability, an agent in production is an unauditable system. It is the most common reason a working pilot stops working and nobody can explain why.
AI agent observability means recording every step of an agent's execution (reasoning, tool calls, retrievals and outputs) and evaluating its quality systematically. It differs from classic monitoring because the failure is not an exception but a plausible yet incorrect decision. Without step-level traces and automated evals, there is no way to know whether an agent is working.
Why an agent isn't debugged like a normal application
In traditional software, a failure produces an error: an exception, a 500, a red test. In an AI agent, the failure produces a well-written wrong answer. No stack trace. No signal.
An agent chains decisions: it interprets the request, picks a tool, invokes it with parameters, reads the result, decides the next step. A real run easily spans 5 to 30 steps. When the final output is bad, the root cause is almost never in the last step: it is in a poor retrieval at the start or a malformed parameter mid-chain.
What agent observability is NOT: it is not storing the prompt and the final answer. That log works for a single LLM call, not for a system making chained decisions. Nor is it classic APM: Datadog tells you the request took 4 seconds, not that the agent picked the wrong tool.
The analogy: debugging an agent without step-level traces is like investigating a plane crash with only the aircraft's final position and no black box.
Three levels: traces, evals and monitoring
These are constantly conflated and they are three different things.
1. Tracing. The structured record of every step of a run: what the agent received, what it reasoned, which tool it called, with what parameters, what came back and how long it took. This is the raw material. Without traces, neither of the other levels exists. The de facto standard in 2026 is OpenTelemetry with GenAI semantic conventions, which avoids vendor lock-in.
2. Evals. The systematic measurement of run quality. Two kinds:
- Offline evals: run against a fixed dataset before deployment. They answer "does this prompt change make things better or worse?".
- Online evals: run on real production traffic, sampled. They answer "is the agent still working well this week?".
3. Operational monitoring. The metrics the business runs on: cost per run, p95 latency, throughput, error rate, token consumption. They answer "is this sustainable at scale?".
| Level | Answers | When it runs | Without it... |
|---|---|---|---|
| Traces | What exactly happened | Always, every run | You cannot diagnose anything |
| Offline evals | Does this change improve? | Before every deploy | You change prompts blind |
| Online evals | Is it still working today? | Sampled real traffic | Silent degradation |
| Monitoring | What does it cost and how slow? | Continuous | Cost and latency surprises |
The mistake we see most: companies that build operational monitoring (they have latency and token dashboards) but have no evals. They know exactly what an agent costs and have no idea whether it is right.
What to actually measure in an agent
Chatbot metrics do not work for agents. These are the ones that matter, ordered by what they usually reveal first:
- Tool selection accuracy. Did it pick the right function? This is failure number one and the cheapest to detect.
- Parameter validity. Did it call the right function with the right arguments? A
find_customerwith the wrong identifier returns valid data about someone else: the most dangerous failure, because it does not look like one. - Retrieval quality (if RAG is involved). Did the retrieved chunks contain the answer? If retrieval fails, everything downstream is noise. Measured with context recall and context precision.
- Groundedness. Is the answer supported by what was retrieved, or did the model fill gaps? This is the metric that catches hallucinations in RAG systems.
- Task completion rate. Did the run finish achieving the goal, or loop, give up, or ask for intervention?
- Steps per task. An agent that needs 22 steps for something that should take 4 is badly designed, even when it is right. It is an early indicator of degradation.
- Cost and p95 latency per completed task. Not per call: per task. It is the only unit comparable to the manual process it replaces.
- Human escalation rate. When the agent hands over control. This should be a target metric, not a failure.
A useful caveat: using an LLM as a judge (LLM-as-a-judge) to score responses is a valid and now standard technique, but it must be calibrated against human labels on a sample. An uncalibrated judge produces numbers that go up while quality goes down.
Key market data
- Agent observability consolidated in 2026 as an engineering discipline in its own right, separate from traditional APM, precisely because agents fail in multi-turn sequences where the cause sits several steps before the symptom (Arize, 2026).
- 46% of organisations cite integration with existing systems as their primary challenge in agent deployments, and the median time-to-value is 5.1 months. Deployments that measure nothing are the ones inflating that median.
- Only 31% of enterprises have at least one agent in production, against 62% experimenting with them. The pilot-to-production jump is, in practice, an observability and control problem.
Our view: the gap between "we're experimenting with agents" and "we have agents in production" is mostly explained by the inability to prove an agent works consistently. That is not proven with demos, it is proven with evals.
The stack: which tools are used in 2026
There is no single answer, but the categories are clear:
- Integrated tracing and eval platforms: Langfuse (open source, self-hostable, the default when GDPR or sensitive data is involved), LangSmith (if you already use LangChain/LangGraph), Braintrust and Arize Phoenix/AX (strong on systematic evaluation).
- General observability with an LLM module: Datadog LLM Observability and similar, if your team already lives there and you want a single pane of glass.
- Lightweight open source: Opik (Comet), MLflow for experiment tracking.
Our pragmatic recommendation: start with OpenTelemetry as the instrumentation layer and a platform that consumes it. Instrumenting against an open standard lets you switch observability vendors without rewriting the agent, and in a market consolidating this fast that is worth more than any specific feature.
If you handle personal or regulated data, the decision simplifies: self-hosted. An agent's traces contain, by definition, everything the agent has seen. That includes customer data. Sending full traces to an external SaaS without assessing data processing is a GDPR problem waiting to happen.
Common use cases
Case 1: Silent degradation after a model change.
- Problem: a support agent starts escalating more cases to humans after a model version update. Nobody connects the two for weeks.
- Solution: online evals with daily sampling and an alert on task completion rate.
- Stack: OpenTelemetry + eval platform + calibrated LLM-as-a-judge.
- Result: the drop is detected in hours, not weeks.
Case 2: Cost spiking without usage growth.
- Problem: token spend rises 40% month over month at the same request volume.
- Solution: steps-per-task metric. The agent had started entering retry loops after an external API change.
- Stack: step-level traces + alert on the steps-per-run percentile.
- Result: the tool returning malformed errors is identified and retry logic fixed.
Case 3: Correct answers over the wrong documents.
- Problem: a RAG agent answers confidently using chunks from the outdated version of a procedure.
- Solution: measure context precision and add a document-version filter to retrieval.
- Stack: retrieval evaluation + version metadata in the vector store.
- Result: the issue is correctly identified as retrieval, not model: which is where everyone was looking.
How to implement it step by step
- Instrument before you scale. Add traces from the first prototype, with OpenTelemetry where possible. Instrumenting an agent already in production costs three times as much and forces changes to live code.
- Build an evaluation dataset from real cases. 30-50 well-chosen cases drawn from real traffic beat 500 synthetic ones. Include the rare cases that have already failed.
- Define what "correct" means for each case. This is the step people skip and without which nothing else works. If you cannot write the success criterion, you cannot automate it.
- Automate offline evals in your deployment pipeline. No prompt, model or tool change ships without passing the dataset. Treat prompts as code.
- Turn on sampled online evals. A share of real traffic evaluated daily. This is what catches degradation offline tests never see.
- Alert on the metrics that matter: task completion dropping, steps per task rising, escalations rising. Do not alert on latency and cost alone.
- Close the loop. Every failure found in production becomes a new case in the evaluation dataset. Without this step the system is observed, not improved.
Common mistakes (and how to avoid them)
Mistake: building observability after the incident. → Reality: by the time the incident arrives you no longer have the traces of what happened. Instrumentation is preventive by definition.
Mistake: logging only input and final output. → Reality: in a 15-step chain, an endpoints-only log says nothing. You need the intermediate step where it went wrong.
Mistake: confusing monitoring with evaluation. → Reality: knowing the agent replies in 2 seconds and burns X tokens says nothing about whether it is right. Different questions, different instruments.
Mistake: using LLM-as-a-judge without calibration. → Reality: a judge never checked against human labels can report 90% accuracy while real users complain. Calibrate on a sample before trusting the number.
Mistake: an evaluation dataset made of easy cases. → Reality: a dataset the agent always passes measures nothing. It must contain ambiguous cases and the ones that already failed.
Mistake: shipping traces with personal data to a SaaS without assessing processing. → Reality: traces contain everything the agent has seen. With regulated data, self-host or anonymise first. This is not a compliance detail, it is the decision that constrains tool choice.
Realistic timelines and effort
- Basic trace instrumentation on an existing agent: 1-2 weeks, longer if the agent was built without a tool abstraction.
- First useful evaluation dataset (30-50 cases with defined success criteria): 1-2 weeks, most of it spent agreeing what "correct" means, not writing code.
- Automated evals in the deployment pipeline: one additional week once the dataset exists.
- Where the real effort goes: not into the tools, which are mature. Into defining the business quality criteria. That is a product conversation, not an infrastructure one.
Metrics to track from day 1: trace coverage (share of fully instrumented runs), task completion rate, steps per task, human escalation rate and cost per completed task. If an agent has been in production for over a month and you cannot state its accuracy on real cases, it is not in production: it is loose.
Frequently asked questions
What is AI agent observability?
It is the practice of instrumenting, tracing and evaluating agentic systems in production, recording every execution step (reasoning, tool calls, retrievals and outputs) to diagnose failures and measure quality. It differs from traditional APM because an agent's failure is not an exception but a plausible yet incorrect decision.
What is the difference between evals and monitoring?
Evals measure whether the agent does its job well (accuracy, groundedness, task completion). Monitoring measures what it costs to operate (latency, tokens, throughput, errors). An agent can have perfect operational metrics while giving wrong answers.
What is an online eval and how does it differ from offline?
An offline eval runs against a fixed dataset before deployment and answers "does this change improve things?". An online eval runs sampled on real traffic and answers "is the agent still working well today?". They catch different problems and you need both.
Can one LLM evaluate another LLM?
Yes, it is the standard technique in 2026 for evaluating at scale, but it must be calibrated against human labels on a sample. An uncalibrated judge produces stable scores that do not match user-perceived quality.
Can I use Datadog or New Relic for AI agent observability?
They cover operational metrics and give you a single pane of glass if your team already lives there, but on their own they do not cover quality evaluation or step-level semantic tracing. The usual pattern is combining the corporate tool with a dedicated eval platform.
How does GDPR apply to agent traces?
Traces contain everything the agent has processed, including personal data. With regulated data, the sensible option is a self-hosted platform or anonymisation before sending. Decide this before choosing a tool, not after.
How many cases does an evaluation dataset need?
Between 30 and 50 well-chosen cases from real traffic give more signal than hundreds of synthetic ones. What matters is that they include ambiguous scenarios and known failures, not the easy cases the agent always solves.
Which metric warns you before an agent degrades?
Steps per task. A rise in the number of steps needed to solve the same thing usually precedes the drop in accuracy and the rise in cost, and shows up before user complaints.
Do you have an agent in production you cannot prove works?
At Naxia we instrument agents with step-level traces, evaluation datasets built from your real cases and automated evals in the deployment pipeline. Using open standards (OpenTelemetry) so you are not locked to a vendor, with a self-hosted option when regulated data is involved.
If you want to move from "it seems to work" to a number you can defend in a steering committee, talk to us. No commitment, no 40-slide decks.
Or explore our implementation process first.