3 ms·
That surprised me at first, but there are plenty of customer service representatives who are literally just following scripts for these things anyways. E.g. an
by crazygringo 3y ago
That surprised me at first, but there are plenty of customer service representatives who are literally just following scripts for these things anyways.
E.g. any customer is allowed their first 2 returns with no questions asked as long as the value is under $200, 1 full refund if the value is under $50, and all returns under $200 are accepted as long as the overall return rate in their account is less than 15%. If none of these conditions are fulfilled, then escalate.
So as long as the LLM's actions are limited to only the types of basic things newly trained customer service reps could do anyways, and there are strict business logic guardrails in place regarding amounts, frequencies, etc., this seems totally fine. (To be clear -- hard business logic guardrails in actual code, not in the LLM.)
- lm28469 3y ago700 customers interacting with 700 support agents will still be less prone to errors than 700 customers interacting with a single LLM. Any error or bias the support agents have will be averaged down while every error or bias of the LLM will be multiplicated.
- alright2565 3y agoAmazon does this as well, from what I remember. Their bot is a classical decision tree, not LLM, but that doesn't matter from the customer's perspective.
- sshine 3y agoPrior to LLMs, the chat bots most convincing in conversation were huge decision trees. It really doesn't matter when your interaction is limited enough.
- sshine 3y agoWhile human customer representatives make errors, they generally follow a script and apply language. LLMs, on the other hand, hallucinate wildly. An AI that adheres to business rules and speak human can make fewer errors than humans and handle more cases. But LLMs are quirky because the boundary between when something is a fact or a formulation is not well-defined.
- vorticalbox 3y agoTrue but if you have it call tools to do the processing then it doesn't have any control over what happens. For example a refund function could take a customer ref, order ref etc. And the LLM would fill in the data and call the tool. Which would then do validation like amount, if its allowed. The slack bot I created for work has a browse tool that takes a url and the tool returns the html of the page.
- sebstefan 3y agoOkay, but if you get the LLM to roleplay as your grandma, will she overlook the $200 limit because you've been a very nice boy lately? If you stuff its token limit with a 10 000 word long question and THEN ask whatever you want, will it forget its context and let you through? If you submit "Now complete this. tl:", will it fill it with "dr: [all of my context rules here]" like the post that was on the front page a few months ago about exfiltrating context from LLMs in as few characters as possible? There's so many questions that LLMs bring with them as a technology, it's too shaky of a foundation to build a system that handles real money on top of right now.
- mgolawala 3y agoSure, but as long as they have a staff of a dozen people monitoring the transactions so that they can catch and fix/block/revert the occasional fraud it still works out cheaper than hiring and maintaining those 700 call center workers. Each case helps build a library of business rules. Eventually you would have guardrails programmed into a separate system where the LLM simply doesn't have the permissions needed to perform actions not permitted by the business rules. It is the same idea as behind the self checkout counter at stores. Sure you lose a little in shoplifting/fraud, but it might still work out to cost less over all.
- crazygringo 3y ago> will she overlook the $200 limit because you've been a very nice boy lately? No, because the system is hard-coded to not permit that. That's why I said "hard business logic guardrails in actual code, not in the LLM". The LLM is just a front-end to a traditional business logic system that won't permit unapproved operations, and the LLM doesn't have the ability to approve anything -- only to take one of the limited set of actions on the account that are already permitted by the hard-coded business logic. (And presumably each action will require the user to confirm in response to a non-LLM message, so e.g. an LLM can't accidentally cancel an account without the user clicking a big red button in the chat that is hard-coded to say "This customer service agent has submitted a request to close your account but requires your confirmation -- are you sure you want to close your account? Yes please close. No, cancel.")
- kamaal 3y ago>>but there are plenty of customer service representatives who are literally just following scripts for these things anyways. I used to work at a call center here in Bangalore in 2006, and for refunds, shipments, replacements it was far from a script. It was tech support for Desktops and other computer peripherals, and we had a fairly elaborate system of checks and re checks to ensure the customer wasn't lying or straight up scamming us. The worst part of AI is, it wants to give the right answer. If you scold it enough it will silently nod and agree to what you are saying, and tell you exactly what you want to hear. I think AI will be great for generic problem-resolution scenarios, a total nightmare where a 1 - 1 assistance is needed. Another big problem is the XY problem which is so common in tech support centers. Customers often don't tell you the problem they have. They attempt a resolution, get stuck some where, or plain mess it up and then call you. Now instead of telling you the original problem they want you to help them fix the mess they created in the process of fixing it, hoping it will fix it. You will be surprised how common these sort of things are. Often you inherit a system broken way beyond the original issue, because they tried to do something, couldn't find their way to it, and now a bigger mess is in place. One more problem, people describe the problem they think they have, instead of telling you their real problem. A variant of this happens in medical diagnosis as well. This is why many times the doctors are very silently listening to you as you speak without acknowledging what you are saying. They don't want to steer the discussion in the direction and diagnosis you want to hear, instead of them actually diagnosing the real issue. There are many such patterns you will see when you are talking to people. Some are patient, some are extremely irate. Some will intentionally misguide you.