Why AI can't replace customer service
And the potential they can unlock together
The Prophecy
The notion that “AI will replace customer service” has become a recurring prophecy over the years and even more in the past few months.
Conversational AIs have existed for years, and we’ve all been in frustrating situations where we get stuck in an endless conversation loop that doesn’t solve our problem. We end up furiously asking to be connected to a human agent.
Our perception of AI has been flipped because of the seemingly-overnight progress spurred by ChatGPT and LLM-powered chatbots.
They don’t feel rigid and limited like their predecessors.
They sound smarter.
They listen carefully to every word you say.
They generate eloquent answers no matter what you ask.
They can talk like a pirate if you want them to.
The sky is the limit!
With this leap in progress, the prophecy seems inevitable: who needs human agents when AI can answer all your customers?
I find this vision of the future overly ambitious because while AI is good, it’s not perfect. It ignores the weaknesses and limitations of these systems and expects too much from them.
People still value human agents
AI’s double-edged sword is that it generates plausible and believable answers to anything, even if it’s not factual. LLMs generate text by predicting the most-likely word in the sequence based on all the previous words it received.
They make predictions rather than writing information with 100% accuracy. And while they may answer correctly most of the time, you can’t guarantee they’ll do that every time.
If I own a business, I create the risk of giving customers the wrong answers or advice if I delegate all my customer service to a chatbot with non-deterministic outputs.
When it comes to customer service, accuracy and adequacy of support are what keep clients in business with you. Lousy service is enough to push people away: “Look at this company! They don’t care enough about customers to have an actual human to help me”.
Intercom, a company that builds customer support tools (including AI chatbots), recently conducted a customer sentiment study on the topic:
- Pre-ChatGPT, 8% of customers used a chatbot in their most recent customer service experience, of which 25% would use it again.
- Post-ChatGPT, customer sentiment around AI chatbots has improved significantly, and conversations feel more “natural” than their predecessors.
- Customers still view chatbots as a barrier to accessing a human agent, which they find valuable.
Being able to convey emotions and frustrations is important for end users, and they want to know there’s an option to deal with a person – often there is an underlying need to feel heard. Frequently, people feel like a human support agent will be able to show flexibility in resolving their issue in a way that a chatbot can’t.
– Intercom, How do your customers feel about AI chatbots?
The potential of AI and human agents working together
Even though AI isn’t perfect, it can still provide immense value.
One experiment at a Fortune 500 company opened my eyes to the potential of AI chatbots and human agents working together.
Instead of creating an AI chatbot for customers, they made one for customer support agents. The chatbot could:
- Monitor chats between human agents and customers
- Give real-time suggestions on how to respond to customers
- Retrieve relevant links and documentation for troubleshooting
Researchers from Stanford and MIT conducted the study with 5,000 human agents and measured the impact after 5 months:
- The company resolved 14% more complaints per hour.
- Agents spent 9% less time per chat and handled 14% more conversations per hour
- Less experienced workers benefitted the most, with productivity gains of 35%.
- Attrition rates were 8.6% lower among agents using the AI chatbot than those without.
- There were little to no negative effects on top-most experienced workers (I couldn’t access the complete study to determine what those little negative effects were).
One important thing to mention: the AI chatbot was trained on the style and outputs of the company’s most productive agents, which explains the positive impact on less experienced workers.
Looking at this experiment, we can see the value of deploying an AI chatbot behind-the-scenes for human agents to use rather than letting customers interact with it and hoping it does a good job.
By keeping humans in the loop, we address AI chatbots’ weaknesses:
- Hallucination → If the chatbot isn’t providing factual information, a human agent can detect that. They’ll ignore its suggestion and provide adequate responses based on their knowledge.
- Accuracy → If the chatbot misunderstands a request or question and retrieves the wrong information, a human agent can also correct that.
- Prompt Engineering → We might need to use specific words and expressions to get the most accurate results. This is a burden that customers don’t want, and human agents are much more versed in a company’s lingo and technical terms to get the best outputs.
Don’t we need fewer human agents then?
When I’ve talked about this to others, one question inevitably comes up: if every human agent is working so much faster with an AI chatbot, won’t they lay off many of them if it maintains the same performance?
There are two other ways of looking at it.
#1: A training tool
In the Fortune 500 company experiment, we saw that less experienced workers had the most to gain from an AI chatbot. In contrast, it had no significant effects on top performers.
The chatbot acted as a training tool to elevate the performance of agents who were still learning and didn’t have as much experience. Usually, the top performers and most experienced people spend significant time helping or training their less experienced counterparts.
They both benefit in this case. Top performers can focus on their work without sacrificing additional time to train and assist others. New employees get to learn from an AI assistant (trained on the style and responses of top performers), which is available 24/7.
From the same study:
An agent using the AI tool who had just two months’ tenure at the firm performed as well as an agent with six months’ tenure working without the tool.
#2: Elevating organizational performance
The other perspective is performance-related.
Sure, a group of human agents with an AI assistant could answer the same number of customers and resolve the same complaints as a bigger group of human agents working alone.
But why match the same performance when we can take it to the next level?
If a company took 3-5 business days to reply, wouldn’t it be amazing if it always answered within a day instead?
AI assistants could create the same output and performance level with fewer people. But maintaining the same number of people will elevate the performance and customer experience.
Ultimately, it comes down to the metric companies try to optimize because it can lead to significantly different outcomes:
- If a company focuses on costs, it will look at eliminating jobs since AI assistants can compensate and make the rest more productive with no performance loss.
- If a company focuses on customer lifetime value, it will keep the same number of people and elevate them with AI assistants to provide unparalleled speed and support.
Who are you going to keep doing business with? The company that still takes 3-5 business days to get back to you, or the one that can answer on the same day?
Some of my thoughts are idealistic, but it’s critical to question trivial solutions that expect too much from technologies like AI. We can unlock tremendous value if we open our minds and explore counterintuitive approaches to solving problems.
Further reading
- Measuring the Productivity Impact of Generative AI (NBER)
- How do your customers feel about AI chatbots? (Intercom)
- AI Improves Employee Productivity by 66% (NNg)
