Most 'AI Agents' in Customer Service Still Operate at Basic Levels, New Analysis Shows

Most 'AI Agents' in Customer Service Still Operate at Basic Levels, New Analysis Shows

A groundbreaking analysis of artificial intelligence agents deployed in real-world products reveals that the vast majority are operating at only the most rudimentary levels, despite the advanced capabilities of underlying language models. This disconnect is leaving users frustrated and questioning the value of AI in customer service.

"The gap between what frontier models can achieve and what users experience in products is enormous," said Dr. Emma Li, an AI researcher at Stanford University. "Most so-called agents are simply wrappers around a single API call, lacking any real context or memory."

Background

The analysis categorizes AI agents into four distinct levels, ranging from basic LLM wrappers to fully autonomous agent loops. The research found that most commercial implementations remain stuck at level 1 or 2, explaining why many users perceive these systems as "dumb" despite the intelligence of the underlying models.

"It's not the model that's the problem," said Mark Chen, CTO of a leading customer service AI company. "The bottleneck is context management and the lack of a proper agent loop. Companies are rushing to market with incomplete solutions."

Level 1: The LLM Wrapper

At the most basic level, an AI agent consists of a simple user input going directly to a language model and returning a response. There are no tools, no memory, no retrieval, and no state management. This configuration handles only the most straightforward FAQ-style questions.

"When a user says 'Check my latest order and cancel it if it hasn't shipped yet,' the seams show immediately," Dr. Li noted. "Nothing is connected to anything. The model is essentially guessing."

This level is where most AI features bolted onto existing SaaS products reside, and it is the primary reason many consumers walk away believing AI is overhyped.

Level 2: Intent Classification Agent

The second level adds a step: the user input first passes through an intent classifier, then routes to an intent-specific handler before generating a response. For customer support, common intents might include refund requests, shipping questions, payment issues, account problems, or escalation to a human.

Within a tightly scoped domain, this approach works surprisingly well. However, the moment a user asks a follow-up question referencing previous information, the system fails. The classifier cannot maintain context across multiple interactions.

"After two messages with one of these systems, you can usually guess the architecture from the outside," Chen added. "The second message reveals everything."

What This Means

These findings have significant implications for businesses deploying AI agents. Without proper context management and agent loops, customer experience suffers, and trust in AI technology erodes. Companies that invest in higher-level agents — those with memory, tool use, and autonomous decision-making — will differentiate themselves.

"The companies that climb to level 3 and 4 will see dramatically better customer satisfaction," Dr. Li predicted. "The gap is not in the model; it's in the architecture."

The full four-level taxonomy includes two higher tiers: context-aware agents with memory and retrieval, and agent loops that can plan, use tools, and iterate. These advanced implementations remain rare in production but offer a roadmap for improvement.

For now, the message to businesses is clear: if your AI agent cannot remember the previous message or take an action on your behalf, it is not an agent at all — it is a wrapper with a marketing label.