Agent Observability Is Becoming Infrastructure. Traces Alone Still Won't Fix Bad Knowledge

Something is hardening in the AI infrastructure market. Not a single announcement—a pattern. Over the past few weeks, Microsoft, AWS, Elastic, IBM, and agent evaluation tooling like DeepEval have all independently landed on the same conclusion: production AI agents need structured traces. Not as a developer convenience. As infrastructure.

That shift is worth paying attention to. But so is what it leaves out.

What's actually happening

Microsoft's Agent Framework now includes built-in OpenTelemetry integration as a core capability. Not a plugin, not an afterthought. Their Foundry tracing documentation specifies what a proper agent trace should capture: inputs and outputs, tool usage, retries, latencies, costs, and token consumption for every run.

On the multi-agent side, Microsoft is extending OpenTelemetry with new semantic conventions for multi-agent systems, in collaboration with Cisco Outshift. The goal is a standard telemetry vocabulary covering quality, performance, safety, cost, tool invocation, and how agents coordinate with each other.

AWS made a similar statement on March 31. Their DevOps Agent, built on Bedrock AgentCore, ships with dedicated infrastructure for memory, policies, evaluations, and observability baked in from the start.

DeepEval has framed its evaluation approach around traces: agent failures should be decomposed into the reasoning layer, the action layer, and overall execution. The trace is the primary unit of analysis.

Elastic's 2026 observability trends report puts numbers to the shift: 85% of organizations already use some form of GenAI for observability, with 98% projected within two years—though only 8% have finished enabling LLM observability specifically. IBM's 2026 observability trends piece reinforces the same: AI systems will require more intelligent observability platforms and stronger open standards.

Even at the individual developer level, you can see the demand. A Hacker News thread around tmux-agent-status—a tool for monitoring coding agents in the terminal—surfaced immediately once people started running agents for hours at a time. The first thing users wanted was visibility into what the agent was actually doing.

This isn't one company's product decision. It's a category hardening.

Why traditional monitoring doesn't work for agents

Classic application monitoring assumes deterministic behavior. You log errors, measure response times, track error rates. The execution path is predictable enough that you can draw a flowchart of what went wrong.

Agents don't work like that. A single agent invocation might involve dozens of tool calls, external retrievals, internal reasoning steps, and retries—some triggered by failures the agent itself decided to handle. A multi-agent workflow adds another layer: agents delegating to other agents, with no single thread to follow.

You can't trace that with standard APM tools. You need something that understands the structure of an agent run: what the agent decided to do and why, which tools it called and in what order, which retrieval steps returned what content, where latency spiked, what the token and cost accounting looks like per step.

This is what OpenTelemetry-based agent tracing is designed to capture. Step replay for debugging. Evaluation frameworks that tie outcomes to specific execution steps. Dashboards that give operations teams visibility into long-running agents without having to dig through logs manually.

For enterprises that have moved beyond pilots, this is a real operational need. You can't run production agents blind.

Where traces stop being enough

Here's what an execution trace actually tells you: the agent used document X to justify action Y at timestamp Z.

It doesn't tell you whether document X was accurate, whether it had been superseded by a newer version, or whether it contradicted something in document Q that the agent had accessed in a previous step.

If an agent quoted a stale reimbursement policy to an employee, a trace records the mistake. It shows you exactly which retrieval step returned the wrong document, which tool call used it, and what output was generated. That's genuinely useful for debugging.

But the trace is an autopsy. It documents what went wrong. It doesn't address why the knowledge was wrong in the first place, and it does nothing to prevent the same mistake on the next run.

DeepEval's breakdown of agent failures into reasoning layer and action layer is correct. What it doesn't address is the layer underneath both: the knowledge that informed the reasoning.

The distinction matters because the failure mode is different. An agent that reasons incorrectly about correct information is an agent problem. An agent that reasons correctly about incorrect information is a knowledge problem. Traces help you see both—but only knowledge governance can fix the second one.

What this means for enterprise AI teams

Enterprise teams deploying production agents are going to acquire observability tooling. That's already clear from the market signals above. The question is what they find when they start using it.

When you can replay an agent run step by step, the first thing you notice is which retrieval calls returned questionable content. Which documents were accessed. Which version. Whether the answer the agent gave matches what the current source actually says.

That's when the next question becomes unavoidable: can we defend the knowledge behind this run?

"What did the agent do" is only half the question. "Was the information the agent relied on current, attributed to an authoritative source, and free of contradictions with other documents in the same knowledge base?" is the other half, and it's the harder one.

This is the gap. Agent observability platforms are excellent at explaining execution. They are not knowledge governance systems. They don't maintain source attribution across a document corpus, detect contradictions between policy documents, or flag when a retrieval returned something outdated.

An enterprise that deploys tracing without governed knowledge underneath gets visible failure—which is better than invisible failure—but not trustworthy execution.

Platforms that get this right treat the knowledge layer as its own infrastructure concern: source attribution on every retrieved chunk, contradiction detection across the document set, scheduled audits for document freshness, and version-aware retrieval. Mojar's architecture is built around exactly this. The observable agent workflow runs on top of that foundation. Without it, the traces explain problems rather than preventing them.

As Mojar AI's approach to RAG reflects: the governed knowledge layer is what makes an agent's actions defensible, not just traceable. That's what AI agent audit trails need to capture, and it's why agent memory infrastructure can't be separated from the question of knowledge accuracy.

What to watch next

More vendors will launch tracing and step-replay features in the next two quarters. The OpenTelemetry semantic conventions for multi-agent systems will drive standardization—or at least a common vocabulary for what traces should capture.

The more interesting development will be pressure on observability stacks to connect execution traces with the knowledge sources that informed them. Right now those are separate concerns. They won't stay separate for long once enterprise teams start using traces to investigate failures and keep arriving at the same root cause: the underlying knowledge was wrong.

Telemetry explains what happened. Governed knowledge explains whether the action was justified.

That's a distinction the market is about to learn the hard way.