Context is King: The Critical Role of RAG in Customer Support AI

Disclaimer: this publication is inspired by the facebook post (in Ukrainian) of Оleksandr Krakоvetskyi. Oleksаndr is CTO at DonоrUA, CEO at DеvRаin and is a Microsoft AI Most Valuable Professional. His post has resonated with me and my thoughts on current AI customer support trends, so I decided to bring this discussion to our audience, using his original publication talking points as they make perfect sense.

As Oleksandr puts it, customer support seems like the perfect candidate for automation and replacing human agents with AI bots. At least, that’s what the majority of businesses believe, and order volumes certainly confirm this trend.

The motivation is clear: potentially eliminate a large number of staff positions while achieving easily calculable economic benefits, unlike other AI implementations where ROI is harder to measure.

So what could possibly go wrong?

When AI Replacement Goes Wrong: Real-World Failures

Klarna’s Expensive Lesson

American financial company Klarna replaced hundreds of customer support employees with artificial intelligence, viewing it as cost optimization and a step into the future.

But the experiment failed spectacularly: customers began complaining en masse about service quality, and the system lost the flexibility and empathy that human agents provided. As a result, the company brought people back, acknowledging that complete automation couldn’t deliver proper customer experience.

Commonwealth Bank’s Voice Bot Disaster

A similar scenario played out at Commonwealth Bank of Australia, which replaced 45 call center employees with a voice bot.

They expected to reduce call volumes but got the opposite effect – the number of inquiries increased, and the system couldn’t handle the load. The bank was forced to apologize, restore jobs, and publicly admit their mistake.

The Context Problem: Why AI Customer Support Fails

The reality is that the majority of customer support inquiries are routine and easily automated. “Majority” doesn’t mean “all.”

Most complex inquiries arise from:

• Billing or accounts issues deliberately made complex
• Internal IT system bugs
• Missing or outdated documentation
• System complexity
• Long customer-company interaction history
• Business processes that lack transparency

Oleksandr: “Try canceling a service, closing an account, switching to a lower plan, or understanding all the nuances of insurance or warranty contracts. Neither artificial intelligence nor humans can quickly navigate these scenarios without proper context.”

TF: Think, for example, of human put into a “chat bot” box, i.e. you have a limited context and no extra connections into the business operations. I.e. you only have the knowledge of the context provided to you by the customer plus limited knowledge base provided to you by the business. You don’t have any direct or informal connections to business operations (an MCP server connection would be super handy here but that’s topic for another discussion.

Add to this completely outdated systems where knowledge bases are maintained (who keeps theirs in markdown format?), plus multiple IT systems that need parallel connections. All of this negatively impacts results.

Rethinking Customer Support’s True Value

Oleksandr: It’s time to fundamentally rethink the meaning of customer support.

First, recognize that customer support communicates with people who are:

• Dissatisfied
• Experiencing frustration
• Close to abandoning your services

A customer whose problem is solved quickly and effectively becomes significantly more loyal. People generally hate dealing with customer support – it’s crucial to understand why and transform this negative into a positive.

Context is Everything: The Missing Link

The issue isn’t that humans are empathetic while AI isn’t – that’s not the core problem. The real issue is that neither humans nor AI often have the necessary context.

Here’s the critical difference: AI won’t gain access to all company context that would be available in its memory anytime soon. Unlike humans, who can store this context informally through experience, relationships, and institutional knowledge.

This is where [a properly configured] RAG (Retrieval Augmented Generation) becomes essential.

The RAG Advantage: Building Context-Aware Support

RAG solves the context problem by ensuring AI systems have access to:

• Real-time company information from multiple sources
• Customer interaction history and previous case details
• Current product documentation and policy updates
• Internal knowledge bases and troubleshooting guides
• System status and known issues

Without proper RAG implementation, your AI support chat bot is essentially guessing based on outdated training data. With RAG, it becomes a knowledgeable agent with access to all the context needed to resolve issues effectively.

Making Customer Support an Asset, Not a Liability

Customer support should be considered an asset, not a liability. The goal should be to:

1. Reduce inquiry volume through better self-service and proactive communication, not just heroically resolve issues after they occur
2. Apply all available technologies to make processing fast, high-quality, and contextual
3. Empower agents (human or AI) with auto-response generation, access to information systems, and most importantly – CONTEXT
4. Focus on prevention rather than just reaction

The Right Approach: Augmentation, Not Replacement

Instead of thinking about eliminating people or replacement, consider augmentation:

• AI handles routine inquiries with full context through RAG
• Complex cases escalate to human agents armed with AI-generated summaries and suggested solutions
• Continuous learning from both AI and human interactions improves the entire system
• Proactive support prevents issues before they become customer complaints

Building Context-Rich Customer Support with Ethora

At Ethora, we understand that successful customer support AI isn’t about replacing humans – it’s about providing both AI and human agents with the context they need to excel.

Our AI Agent Widget with advanced RAG capabilities (such as website URL crawler and RAG documents indexer) ensures your support system has access to:

• Current website content and documentation
• Product information and policy updates
• Knowledge base documents and other context supplementing documents that are not published online
• Historical customer interaction data
• Real-time system status and known issues

The result? Faster resolution times, higher customer satisfaction, and support agents (AI and human) who can actually help instead of just apologising for not having the right information.

Remember: the goal isn’t to eliminate customer support costs, but to transform customer support from a cost center into a competitive advantage. When customers leave interactions feeling genuinely helped rather than frustrated, that’s when support becomes a true business asset.

You can use the embeddable HTML widget or WordPress plugin from Ethora, both coming with RAG features, to validate the RAG-powered customer support chat bot for your business.

---

Keep Reading

• Making your AI agent context-aware with RAG and vector embeddings
• Case study — RAG AI chatbot for a health supplements business
• Why your website’s AI chatbot is probably useless (and how to fix it)

Build a production-grade RAG support bot — explore the Ethora AI SDK, try the agent chatbot builder, or start building for free.

More from Ethora: self-hosted LLM AI agent, healthcare chat SDK.