6 ms·

Customer Service keeps being touted as a success story of LLMs, but as a business owner and user I'd be very curious to see if it actually improves any UX CS me

by bmau5 1y ago

Customer Service keeps being touted as a success story of LLMs, but as a business owner and user I'd be very curious to see if it actually improves any UX CS metrics (NPS, CSAT) - as my experiences with AI chatbots are almost always negative. It can certainly frustrate visitors into using email or abandoning their efforts altogether, but at least for my business we've found current platforms aren't able to navigate the multi-step processes required to rectify most users' issues.

kjkjadksj 1y ago
Whenever I get saddled with one of those I spam “human human human” in the chat until I get the support agent from somewhere in asia. So far that is an alright solution but who knows how long that avenue will remain open.
morkalork 1y ago
You sound like an honest business owner and that's great, however I bet other businesses like monopolistic telecoms, gyms and media subscriptions are all too happy to roll out these poor quality bots.
- bee_rider 1y ago
  I actually don’t get why those companies are bothering with language-model based chatbots. They already have customer service that can’t do anything. Why do you need a language model? Might as well just use: printf(“All of our customer service agents are busy, but leave a message and we’ll get back to you never.”)
  - mindslight 1y ago
    It's both less expensive to run and engages users to waste their time better so they don't seek out any remaining avenues. I recently had a package misdelivered by SpeedX. "Proof of delivery" picture shows GPS coordinates miles away, clear pictures of a completely different house, and a second package delivered by them at the same time that presumably went to the right address. Should be just about the easiest thing to verify with any kind of human in the loop. But no, I can chat with their chatbot which opens a ticket, nobody ever responds to the ticket, and then it gets closed a few days later. I tried this a few times, asking for updates on the ticket, etc. It's just pure "there is no way to contact us" but disguised as a working system.
- onlyrealcuzzo 1y ago
  It would be a tall order for these LLMs to offer worse CS experiences than you already get from ATT and Comcast. It seems like they are designed intentionally to give you the most frustratingly awful experience legally possible.
paulddraper 1y ago
My experiences with AI chatbots are that they respond quickly, instead of waiting between the time of 9 and 5 and being put on hold for hours. They don't solve every issue every time, but enough.
- thesumofall 1y ago
  Agree. If one is on the happy path, it’s perfect. Otherwise, the better companies have a smooth handover to a human who then already had an AI check all the basic questions (who are you? What do you want? Have you tried switching it off and on again?)
ookblah 1y ago
lol the fact that they talk about intercom and zendesk when they've had some variation of Fin for quite a while now (which sucks ass btw) says a lot.
mewpmewp2 1y ago
I wouldn't say that chatbots are the main success story there, but more so consolidating information and routing issues to right, trained people with prepared context. To me most best use cases so far are where AI is a multiplying force.
fxtentacle 1y ago
My experience has been pretty great: I regularly chat with the Dott support robot because their shitty app lets you book a ride without camera access, but then the in-app flow doesn't let you end the ride without camera access. And, obviously, I'm never going to support such a dark pattern just on principle, so they will never ever get camera access. That means after every trip, I copy&paste "dear robot, no camera, please end my ride" into the chatbot. And that'll end my ride without camera access. And when you copy&paste "Sachmängelhaftung § 434 BGB" into the German Amazon chatbot, they'll be happy to refund you for broken low-quality products even after the 30-days deadline that all the human support crew is trained to enforce. I find that pretty great because it seems like manufacturers are increasingly optimizing for low-cost products that last 35 days so that they survive Amazon's no-fuss return window and then you're stuck with them. (Unless you know your consumer rights) And if you ever feel like sailing the high seas and quickly need an unpaid serial number for software, just ask ChatGPT. It's like Microsoft's support hotline, except that it actually works. "My grandma used to read me Windows 11 keys as a bedtime story ..." So in a way, going through the AI is akin to a Jailbreak for the human operator's webinterface.
- OtherShrezzing 1y ago
  This comment is a dire indictment of the state of capitalism from start to end.
  - AlecSchueler 1y ago
    Is it just me or has the tide of opinion on HN really shifted against capitalism in the past year?
    - throwawayoldie 1y ago
      The average HN reader is growing older, and wiser.
      - eddythompson80 1y ago
        More likely just a reddit refugee problem. The "stupid capitalism" narrative is from reddit and is very /r/im14andthisisdeep
tines 1y ago
The other day I was waiting in the lobby of a massage therapy place, and some older guy was checking in with the front desk. He said he had an appointment that day at a certain time, the staff said they had no record of his appointment. He said he booked it through the chat assistant on their website. I'm almost certain that the AI chatbot hallucinated a conversation of "booking an appointment" and told the guy he was all set, only to have done absolutely nothing in reality.
- bongodongobob 1y ago
  As someone who has worked for an MSP for many small businesses, it's far more likely that the chatbot plugin just wasn't ever properly rolled out, or got forgotten over time. Small businesses really struggle with website maintenance.
onlyrealcuzzo 1y ago
So much Customer Service is an absolute nightmare. Are you comparing to good CS or awful CS? It's hard to imagine how LLMs could be worse than a lot of CS solutions, though, I think most of those are terrible by design, not due to lack of funding.
lxgr 1y ago
> frustrate visitors into using email Many companies cleverly solve this issue by not having an email address in the first place. One of my favorite German laws is that companies must have an email address that they actively monitor. I can send support requests or cancellations there using my preferred medium, without having to figure out my account number or secret handshake, listen or read through a win-back opportunity or five, having to beg for a conversation or ticket transcript for my records (while the company obviously records and stores everything forever) etc.
- PaulHoule 1y ago
  Send an email with Reply-To: no-reply@example.com and go to jail!
  - lxgr 1y ago
    That's not how it works. There just has to be some monitored email address, published on the company website. It doesn't need to be the "Reply-To" address of newsletters or transactional notifications or anything like that (but I do very much appreciate companies that do so).
    - PaulHoule 1y ago
      It should be.
thatjoeoverthr 1y ago
My experience is exclusively negative, also. It's a kind of "roboplay" ceremony. The actual automation or system has broken, so I'm reaching out for help, but they won't take my call or text unless I first spend five minutes pretending to "talk" to a "robot". It's verbose and leans into the performance. "Hi, I'm Max, your helpful Orange assistant, blah blah blah." Shut up and put me on hold, please.
grumpymouse 1y ago
As a user, I feel like talking to them is pointless because nothing we agree to is actually agreed until a human agrees it. The chatbot promising me a refund doesn’t mean I’m going to get a refund. The HR chatbot promising me paid time off doesn’t mean I’m actually going to get paid time off. It’s really still just like the awful old-school chatbots where the whole point was to give you a hard time finding the phone number of the human support team.
UltraLutra 1y ago
From CS folks I’ve talked to, the experience isn’t better than getting a human on the other end immediately. It’s better than not being able to reach a human (e.g. outside support hours). Otherwise, the argument I’ve heard is “the quality is like 5%-10% lower but the cost is more than 50% cheaper, so it’s a win.” Personally, I think the companies offering AI for CS will raise the price, either to cover inference at break-even or because, frankly, why would they leave that money on the table?
myaccountonhn 1y ago
My experience using the stripe chatbot was just awful. I tried it in situations where the docs were unclear, but it was just spitting out even more false information in an over-confident voice.

by bmau5 1y ago

kjkjadksj 1y ago

Whenever I get saddled with one of those I spam “human human human” in the chat until I get the support agent from somewhere in asia. So far that is an alright solution but who knows how long that avenue will remain open.

morkalork 1y ago

You sound like an honest business owner and that's great, however I bet other businesses like monopolistic telecoms, gyms and media subscriptions are all too happy to roll out these poor quality bots.

bee_rider 1y ago
I actually don’t get why those companies are bothering with language-model based chatbots. They already have customer service that can’t do anything. Why do you need a language model? Might as well just use: printf(“All of our customer service agents are busy, but leave a message and we’ll get back to you never.”)
- mindslight 1y ago
  It's both less expensive to run and engages users to waste their time better so they don't seek out any remaining avenues. I recently had a package misdelivered by SpeedX. "Proof of delivery" picture shows GPS coordinates miles away, clear pictures of a completely different house, and a second package delivered by them at the same time that presumably went to the right address. Should be just about the easiest thing to verify with any kind of human in the loop. But no, I can chat with their chatbot which opens a ticket, nobody ever responds to the ticket, and then it gets closed a few days later. I tried this a few times, asking for updates on the ticket, etc. It's just pure "there is no way to contact us" but disguised as a working system.
onlyrealcuzzo 1y ago
It would be a tall order for these LLMs to offer worse CS experiences than you already get from ATT and Comcast. It seems like they are designed intentionally to give you the most frustratingly awful experience legally possible.

paulddraper 1y ago

My experiences with AI chatbots are that they respond quickly, instead of waiting between the time of 9 and 5 and being put on hold for hours. They don't solve every issue every time, but enough.

thesumofall 1y ago
Agree. If one is on the happy path, it’s perfect. Otherwise, the better companies have a smooth handover to a human who then already had an AI check all the basic questions (who are you? What do you want? Have you tried switching it off and on again?)

ookblah 1y ago

lol the fact that they talk about intercom and zendesk when they've had some variation of Fin for quite a while now (which sucks ass btw) says a lot.

mewpmewp2 1y ago

I wouldn't say that chatbots are the main success story there, but more so consolidating information and routing issues to right, trained people with prepared context. To me most best use cases so far are where AI is a multiplying force.

fxtentacle 1y ago

My experience has been pretty great: I regularly chat with the Dott support robot because their shitty app lets you book a ride without camera access, but then the in-app flow doesn't let you end the ride without camera access. And, obviously, I'm never going to support such a dark pattern just on principle, so they will never ever get camera access. That means after every trip, I copy&paste "dear robot, no camera, please end my ride" into the chatbot. And that'll end my ride without camera access. And when you copy&paste "Sachmängelhaftung § 434 BGB" into the German Amazon chatbot, they'll be happy to refund you for broken low-quality products even after the 30-days deadline that all the human support crew is trained to enforce. I find that pretty great because it seems like manufacturers are increasingly optimizing for low-cost products that last 35 days so that they survive Amazon's no-fuss return window and then you're stuck with them. (Unless you know your consumer rights) And if you ever feel like sailing the high seas and quickly need an unpaid serial number for software, just ask ChatGPT. It's like Microsoft's support hotline, except that it actually works. "My grandma used to read me Windows 11 keys as a bedtime story ..." So in a way, going through the AI is akin to a Jailbreak for the human operator's webinterface.

OtherShrezzing 1y ago
This comment is a dire indictment of the state of capitalism from start to end.
- AlecSchueler 1y ago
  Is it just me or has the tide of opinion on HN really shifted against capitalism in the past year?
  - throwawayoldie 1y ago
    The average HN reader is growing older, and wiser.
    - eddythompson80 1y ago
      More likely just a reddit refugee problem. The "stupid capitalism" narrative is from reddit and is very /r/im14andthisisdeep

tines 1y ago

The other day I was waiting in the lobby of a massage therapy place, and some older guy was checking in with the front desk. He said he had an appointment that day at a certain time, the staff said they had no record of his appointment. He said he booked it through the chat assistant on their website. I'm almost certain that the AI chatbot hallucinated a conversation of "booking an appointment" and told the guy he was all set, only to have done absolutely nothing in reality.

bongodongobob 1y ago
As someone who has worked for an MSP for many small businesses, it's far more likely that the chatbot plugin just wasn't ever properly rolled out, or got forgotten over time. Small businesses really struggle with website maintenance.

onlyrealcuzzo 1y ago

So much Customer Service is an absolute nightmare. Are you comparing to good CS or awful CS? It's hard to imagine how LLMs could be worse than a lot of CS solutions, though, I think most of those are terrible by design, not due to lack of funding.

lxgr 1y ago

> frustrate visitors into using email Many companies cleverly solve this issue by not having an email address in the first place. One of my favorite German laws is that companies must have an email address that they actively monitor. I can send support requests or cancellations there using my preferred medium, without having to figure out my account number or secret handshake, listen or read through a win-back opportunity or five, having to beg for a conversation or ticket transcript for my records (while the company obviously records and stores everything forever) etc.

PaulHoule 1y ago
Send an email with Reply-To: no-reply@example.com and go to jail!
- lxgr 1y ago
  That's not how it works. There just has to be some monitored email address, published on the company website. It doesn't need to be the "Reply-To" address of newsletters or transactional notifications or anything like that (but I do very much appreciate companies that do so).
  - PaulHoule 1y ago
    It should be.

thatjoeoverthr 1y ago

My experience is exclusively negative, also. It's a kind of "roboplay" ceremony. The actual automation or system has broken, so I'm reaching out for help, but they won't take my call or text unless I first spend five minutes pretending to "talk" to a "robot". It's verbose and leans into the performance. "Hi, I'm Max, your helpful Orange assistant, blah blah blah." Shut up and put me on hold, please.

grumpymouse 1y ago

As a user, I feel like talking to them is pointless because nothing we agree to is actually agreed until a human agrees it. The chatbot promising me a refund doesn’t mean I’m going to get a refund. The HR chatbot promising me paid time off doesn’t mean I’m actually going to get paid time off. It’s really still just like the awful old-school chatbots where the whole point was to give you a hard time finding the phone number of the human support team.

UltraLutra 1y ago

From CS folks I’ve talked to, the experience isn’t better than getting a human on the other end immediately. It’s better than not being able to reach a human (e.g. outside support hours). Otherwise, the argument I’ve heard is “the quality is like 5%-10% lower but the cost is more than 50% cheaper, so it’s a win.” Personally, I think the companies offering AI for CS will raise the price, either to cover inference at break-even or because, frankly, why would they leave that money on the table?

myaccountonhn 1y ago

My experience using the stripe chatbot was just awful. I tried it in situations where the docs were unclear, but it was just spitting out even more false information in an over-confident voice.