"Ask Me Anything": How Comcast Uses LLMs to Assist Agents in Real Time

• Main Authors: Rome, Scott, Chen, Tianwen, Tang, Raphael, Zhou, Luwei, Ture, Ferhan
• Format: Journal Article
• Language: English
• Published: 06-05-2024
• Subjects: Computer Science - Computation and Language
• DOI: 10.48550/arxiv.2405.00801

Customer service is how companies interface with their customers. It can
contribute heavily towards the overall customer satisfaction. However,
high-quality service can become expensive, creating an incentive to make it as
cost efficient as possible and prompting most companies to utilize AI-powered
assistants, or "chat bots". On the other hand, human-to-human interaction is
still desired by customers, especially when it comes to complex scenarios such
as disputes and sensitive topics like bill payment.
This raises the bar for customer service agents. They need to accurately
understand the customer's question or concern, identify a solution that is
acceptable yet feasible (and within the company's policy), all while handling
multiple conversations at once.
In this work, we introduce "Ask Me Anything" (AMA) as an add-on feature to an
agent-facing customer service interface. AMA allows agents to ask questions to
a large language model (LLM) on demand, as they are handling customer
conversations -- the LLM provides accurate responses in real-time, reducing the
amount of context switching the agent needs. In our internal experiments, we
find that agents using AMA versus a traditional search experience spend
approximately 10% fewer seconds per conversation containing a search,
translating to millions of dollars of savings annually. Agents that used the
AMA feature provided positive feedback nearly 80% of the time, demonstrating
its usefulness as an AI-assisted feature for customer care.