Headline

AI Agents Drive Up Telemetry Costs and Complicate Network Observability

Key Facts

• A survey of 2,050 IT operations and engineering professionals in 10 countries found that 57% of organizations have observability costs that exceed the cost of the infrastructure being monitored.
• 60% say unexpected billing spikes create more operational strain than consistently high costs; 55% cannot accurately control AI expenses.
• A production AI agent can emit four to five times more telemetry data than traditional applications, with multi-agent sessions increasing volume exponentially.
• Only 6% name AI infrastructure as their biggest observability cost driver; noisy logs (31%), high-cardinality metrics (27%) and trace volumes (21%) top the list.
• 38% have reduced telemetry coverage or retention to control costs, potentially creating troubleshooting blind spots.
• 87% say monitoring AI is a priority, but 56% cannot measure AI workload metrics such as model response quality, 49% say tools lack adequate AI monitoring, and 38% have not clearly defined who owns AI monitoring.
• 81% report that monitoring AI workloads has increased collaboration between observability and security teams.

A new industry survey paints a difficult picture for enterprise technology teams: AI agents are becoming both a major source of machine data and a major consumer of observability tooling. The result is a sharp rise in telemetry costs, more complicated network management and a growing risk that cost-control measures will leave operations teams without the data they need when problems occur.

The research, based on responses from 2,050 IT operations and engineering professionals across 10 countries, found that observability spending has reached an uncomfortable level for many organizations. Fifty-seven percent said their observability costs exceed the cost of the infrastructure being monitored. That statistic matters because observability is supposed to be a supporting capability, not a larger line item than the systems it is meant to protect.

Cost volatility is also a serious concern. Sixty percent of respondents said unexpected billing spikes create more operational strain than consistently high costs. Predictable expenses can be planned for; sudden increases force teams to renegotiate budgets, delay projects or cut data collection. Fifty-five percent said they cannot accurately control their AI expenses, a sign that AI adoption is outpacing the financial controls and governance models many organizations have in place.

Why AI Agents Generate So Much Telemetry

Traditional applications produce logs, metrics and traces around requests, errors and latency. AI agents add another layer. A production agent can emit four to five times more telemetry data than a conventional application. That data can include workflow steps, tool calls, retrieved context, model interactions, authentication events and the results of decisions made during a task.

The problem compounds when agents interact with other agents. Multi-agent sessions can create exponential growth in telemetry because each agent may log its own actions, inputs, outputs and handoffs. A single user request can trigger a chain of model calls, data retrievals, policy checks and external tool invocations. Every step produces data that may be useful for debugging, compliance, performance tuning or security analysis. Collecting all of it, however, can be expensive and operationally complex.

AI agents are not only producers of machine data; they are also consumers. Agents use observability data to reason about system state, diagnose issues or decide what action to take next. This two-way relationship accelerates data volume. The more agents are deployed, the more telemetry is generated, and the more telemetry the agents may need to function effectively. Without careful design, that feedback loop can quickly become a budget problem.

The Cost Drivers Are Broader Than AI

The survey offers an important nuance: AI infrastructure is not yet the single largest observability cost driver. Only 6 percent of respondents named it as their biggest expense. Noisy logs topped the list at 31 percent, followed by high-cardinality metrics at 27 percent and trace volumes at 21 percent. This suggests that AI agents are adding pressure to an already strained telemetry pipeline rather than creating an entirely new problem from scratch.

Many organizations have long collected more data than they need. Verbose application logs, detailed metrics with many unique labels and high-volume traces can produce substantial storage and processing bills. AI agents make that existing problem worse by adding new categories of data and by increasing the rate at which data is produced. The research did not quantify exactly how much of the cost increase comes from AI agents versus traditional collection practices, but the directional effect is clear: agents increase volume, variety and unpredictability.

Cost Controls Create Blind Spots

When budgets tighten, teams often reduce telemetry coverage or retention. The survey found that 38 percent of organizations have already done so to control costs. That decision can create blind spots for network and application troubleshooting. Historical investigations may become impossible when retention windows are shortened. Reduced sampling can increase the risk of missing intermittent problems. One-off troubleshooting often depends on data that was not expected to be valuable at the time it was collected.

For network teams, this is especially risky. Networks are dynamic, and problems can appear only under specific combinations of traffic, configuration and load. If telemetry is sampled too aggressively or retained for too short a period, teams may lack the evidence needed to identify root causes. They may see that a problem occurred but not why it occurred, or they may miss the early signals that would have prevented a larger outage.

The challenge is not simply to collect less data. It is to collect the right data, keep it for the right amount of time and make it available to the teams that need it. That requires a data strategy rather than a blanket cost-cutting exercise. Teams need to understand which logs, metrics and traces are essential for service-level objectives, security investigations, compliance requirements and incident response. They also need to identify data that is redundant, low-value or duplicative.

Monitoring AI Quality and Safety

Cost is only one part of the observability challenge. Organizations also need to monitor AI agents to ensure they operate correctly and safely. Eighty-seven percent of respondents said monitoring AI is a priority. Yet the survey identified significant gaps. Fifty-six percent cannot measure AI workload metrics such as large language model response quality. Forty-nine percent said their observability tools lack adequate AI monitoring capabilities. Thirty-eight percent said their organizations have not clearly defined responsibility for monitoring AI.

Traditional monitoring tools are good at detecting whether a service is available, whether latency is within tolerance and whether error rates are rising. They are less effective at judging whether an AI-generated answer is accurate, relevant, safe or consistent with policy. Tools alone cannot determine whether an output is good. That requires an agreement on what quality means for each use case, along with the skills to turn that definition into accurate evaluations.

For example, a customer service agent may need to be evaluated on factual accuracy, tone, policy compliance and escalation behavior. A coding assistant may need to be evaluated on correctness, security and maintainability. An internal research agent may need to be assessed on source quality, recency and completeness. Each use case requires different metrics, test data and review processes. Without clear ownership, those evaluations may fall between teams.

Network and Security Teams Share the Burden

Network operations teams have an important role because AI agents generate significant network traffic. Agents call APIs, retrieve data, communicate with other agents and interact with external services. Teams must continue to monitor traditional metrics such as latency, packet loss, throughput and error rates. They also need to work with security teams to distinguish legitimate agent activity from malicious traffic.

This is not always straightforward. An AI agent repeatedly calling an API could generate traffic that resembles an attack. A multi-agent workflow might create sudden bursts of requests that look like a denial-of-service pattern. Security and observability teams need shared context so they can tell the difference between expected automation and hostile behavior. The survey found that 81 percent of respondents said monitoring AI workloads has increased collaboration between observability and security teams. That collaboration is likely to become more important as agents take on more autonomous tasks.

Practical Steps for Controlling Telemetry Costs

One recommended approach is to deploy a vendor-neutral telemetry collector. A collector can process and export telemetry data, and its processors can be used to manage costs and tag traffic sources. Tagging helps teams understand which agents, applications or infrastructure components are generating data. That visibility is essential for chargeback, capacity planning and targeted cost reduction.

Organizations can also apply filtering, aggregation and sampling at the point of collection rather than after data has been stored and indexed. They can set retention policies by data type, service tier and regulatory requirement. They can reduce noisy logs by standardizing log levels and removing redundant fields. They can control high-cardinality metrics by limiting unnecessary labels. They can use tail-based sampling to keep traces that matter, such as those involving errors or high latency, while discarding routine traces that add little diagnostic value.

Budget predictability matters as much as absolute cost. Unexpected billing spikes cause more strain than consistently high costs, according to 60 percent of respondents. Teams need forecasting, alerts and spending controls tied to telemetry volume. They need to know which teams, applications and agents are responsible for cost changes. They also need procurement and finance teams involved early, so that observability spending is treated as part of the cost of AI operations rather than an afterthought.

As AI agents become more common, the core observability question is changing. It is no longer just about collecting enough data to see what happened. It is about deciding how much data is needed to maintain visibility without allowing costs to spiral out of control. The organizations that succeed will be those that treat telemetry as a managed asset, define quality and ownership for AI monitoring, and build collaboration between network, observability and security teams before the next billing spike or troubleshooting blind spot appears.

Source:Network World News