3 ms·

It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. The bar LLMs have to clear to be

It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. The bar LLMs have to clear to beat the average front line support operations isn't that high, as your own experience shows. And compared to a large force of badly paid humans with high turnover, LLMs are pretty consistent and easy to train to an adequate level. They won't beat great costumer support agents, but most companies don't have many of those

AdamJacobMuller 3y ago
The bar isn't even that high. They only need to increase the lawsuit/settlement amount by less than the amount the companies saves by automation.
itsoktocry 3y ago
>It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuit A human will be more likely to say "I don't know" or pass you along, rather than outright lie.
- deleted 3y ago
  [deleted]
- mmh0000 3y ago
  Maybe... Over two decades ago (holy crap, I'm old), I used to work in the call center for a major airline. I was the guy you got when you complained to the 1st level rep that you wanted their manager, then you got sent to the 2nd level rep and wanted their manager. And that was me. 90% of my job was undoing and compensating passengers for the incorrect information either the phone agent or gate agent gave them. The other 10% was dealing with workarounds to technical issues in our booking software.
- KiranRao0 3y ago
  I find it common for human customer support people to give inaccurate information. I don't know about "outright lying", but I've had people tell me things that are factually incorrect.
  - atoav 3y ago
    Depends. Saying "thing X should not fail" is factually incorrect, when you called because thing X failed. However I would not expect an airline customer support to make up a completely fictional flight that has never existed. Maybe they could confuse flights or read a number wrong, but making one up?
    - erhaetherth 3y ago
      Humans won't fabricate too much but when confronted with yes/no questions and they have a 50-50 shot of being right and any blowback will likely be on someone else....they'll answer whatever to get you out of their hair. Case in point, I asked my bank if they had any FX conversion fees or markup. Guy said no. I asked if there was any markup on the spread. Said no. Guess what? They absolutely mark up that spread. Their exchange rates are terrible. Just because there isn't a line-item with a fee listed doesn't mean there isn't a hidden fee in there. He's either incompetent or a liar.
      - deleted 3y ago
        [deleted]
mynameisvlad 3y ago
Sure, but there is such a thing as “the human element”. Humans aren’t perfect, and that is the expectation. That is not the same case with computers. And especially for something where it’s just pulling data from an internal system. There is absolutely no reason to invent made up information and saying “well humans do it all the time” is just an excuse.
- steveBK123 3y ago
  Yes, further, expectations wise.. On the phone with a customer service rep, I might understand a little wishy washy answer, slip of the tongue or slightly inaccurate statement. I've never really had a rep lie to me, usually its just I don't knows & them escalation as needed. There is something about the written word from a company that makes it feel more like "binding statement".
SOLAR_FIELDS 3y ago
To me that’s totally fine. I don’t even particularly care whether the LLM is better or not. The only thing they really matters is if you are gonna use that LLM, when it inevitably messes up you don’t get to claim that it wasn’t your fault because computers are hard and AI is an emerging field. IDGAF. Pay up. The fact that you dabble in emerging technologies doesn’t give you any excuse to provide lesser services
- steveBK123 3y ago
  Right whether you employ a person or some software which lies, the liability should be the same.
steveBK123 3y ago
It's still way too easy to send LLMs into a complete tangent of rambling incoherently, opening yourself up to the LLM making written statements to customers you really don't want. I recently asked some LLMs "How many gallons in a mile?" and got some very verbose answers, which turned into feats of short story short stories when I refined to "How many gallons of milk in a mile?"
- fennecfoxy 3y ago
  Only because the models have seemingly only been trained on generating text that matches a prompt, ie prompt completion. Rather than knowledge retrieval/parsing/organisation. If part of the training was to only use knowledge sourced from a vector db and that it is allowed to use its trained knowledge only for grammar rules, phrasing or rewriting information then I think it would do a lot better. Doesn't seem like many models are trained on prompts like "Question Q"->"[no data] I'm sorry but I don't know that" = accepted during training. This would help immensely for not just for chatbots but for personal use too. I don't want my LLM assistant to invent a trip to Mars when I ask it "what do I have to do today" and my calendar happens to be empty.
- vGPU 3y ago
  I just tried the latter with gab’s AI and it was excellent.
behringer 3y ago
I think you're underestimating the quality of customer support. People are going to be out there testing every flaw in the support system, staffed or unstaffed. LLMs have no hope.
Wowfunhappy 3y ago
> It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. If a company representative told me in writing (perhaps via email support) that I could claim a refund retroactively, and that turned out to not be their policy, I would still expect the company to honor what I was told in the email. Phone calls are difficult more because there is no record of what was said. But if I had a recording of the phone call... I'm not actually sure what I would expect to happen. It's just so socially unusual to record phone calls.
- epcoa 3y ago
  > It's just so socially unusual to record phone calls. Is it? I can not remember the last time I called some business where I did not get a “this call may be monitored or recorded for quality and training purposes…”. whatever perceived social hangups the company had they got over them and you don’t even need to ask in a 2PC jurisdiction, it’s already taken care of, just record the call.
  - araes 3y ago
    Which implies most humans calling any form of customer service should probably have a local audio recording, because otherwise you're in a lopsided witness situation. Don't want there to be a recording? That call just happened to not be recorded. Notably, this is getting difficult anyway with how easy it is to manufacture vocal data.

AdamJacobMuller 3y ago

The bar isn't even that high. They only need to increase the lawsuit/settlement amount by less than the amount the companies saves by automation.

itsoktocry 3y ago

>It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuit A human will be more likely to say "I don't know" or pass you along, rather than outright lie.

deleted 3y ago
[deleted]
mmh0000 3y ago
Maybe... Over two decades ago (holy crap, I'm old), I used to work in the call center for a major airline. I was the guy you got when you complained to the 1st level rep that you wanted their manager, then you got sent to the 2nd level rep and wanted their manager. And that was me. 90% of my job was undoing and compensating passengers for the incorrect information either the phone agent or gate agent gave them. The other 10% was dealing with workarounds to technical issues in our booking software.
KiranRao0 3y ago
I find it common for human customer support people to give inaccurate information. I don't know about "outright lying", but I've had people tell me things that are factually incorrect.
- atoav 3y ago
  Depends. Saying "thing X should not fail" is factually incorrect, when you called because thing X failed. However I would not expect an airline customer support to make up a completely fictional flight that has never existed. Maybe they could confuse flights or read a number wrong, but making one up?
  - erhaetherth 3y ago
    Humans won't fabricate too much but when confronted with yes/no questions and they have a 50-50 shot of being right and any blowback will likely be on someone else....they'll answer whatever to get you out of their hair. Case in point, I asked my bank if they had any FX conversion fees or markup. Guy said no. I asked if there was any markup on the spread. Said no. Guess what? They absolutely mark up that spread. Their exchange rates are terrible. Just because there isn't a line-item with a fee listed doesn't mean there isn't a hidden fee in there. He's either incompetent or a liar.
    - deleted 3y ago
      [deleted]

mynameisvlad 3y ago

Sure, but there is such a thing as “the human element”. Humans aren’t perfect, and that is the expectation. That is not the same case with computers. And especially for something where it’s just pulling data from an internal system. There is absolutely no reason to invent made up information and saying “well humans do it all the time” is just an excuse.

steveBK123 3y ago
Yes, further, expectations wise.. On the phone with a customer service rep, I might understand a little wishy washy answer, slip of the tongue or slightly inaccurate statement. I've never really had a rep lie to me, usually its just I don't knows & them escalation as needed. There is something about the written word from a company that makes it feel more like "binding statement".

SOLAR_FIELDS 3y ago

To me that’s totally fine. I don’t even particularly care whether the LLM is better or not. The only thing they really matters is if you are gonna use that LLM, when it inevitably messes up you don’t get to claim that it wasn’t your fault because computers are hard and AI is an emerging field. IDGAF. Pay up. The fact that you dabble in emerging technologies doesn’t give you any excuse to provide lesser services

steveBK123 3y ago
Right whether you employ a person or some software which lies, the liability should be the same.

steveBK123 3y ago

It's still way too easy to send LLMs into a complete tangent of rambling incoherently, opening yourself up to the LLM making written statements to customers you really don't want. I recently asked some LLMs "How many gallons in a mile?" and got some very verbose answers, which turned into feats of short story short stories when I refined to "How many gallons of milk in a mile?"

fennecfoxy 3y ago
Only because the models have seemingly only been trained on generating text that matches a prompt, ie prompt completion. Rather than knowledge retrieval/parsing/organisation. If part of the training was to only use knowledge sourced from a vector db and that it is allowed to use its trained knowledge only for grammar rules, phrasing or rewriting information then I think it would do a lot better. Doesn't seem like many models are trained on prompts like "Question Q"->"[no data] I'm sorry but I don't know that" = accepted during training. This would help immensely for not just for chatbots but for personal use too. I don't want my LLM assistant to invent a trip to Mars when I ask it "what do I have to do today" and my calendar happens to be empty.
vGPU 3y ago
I just tried the latter with gab’s AI and it was excellent.

behringer 3y ago

I think you're underestimating the quality of customer support. People are going to be out there testing every flaw in the support system, staffed or unstaffed. LLMs have no hope.

Wowfunhappy 3y ago

> It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. If a company representative told me in writing (perhaps via email support) that I could claim a refund retroactively, and that turned out to not be their policy, I would still expect the company to honor what I was told in the email. Phone calls are difficult more because there is no record of what was said. But if I had a recording of the phone call... I'm not actually sure what I would expect to happen. It's just so socially unusual to record phone calls.

epcoa 3y ago
> It's just so socially unusual to record phone calls. Is it? I can not remember the last time I called some business where I did not get a “this call may be monitored or recorded for quality and training purposes…”. whatever perceived social hangups the company had they got over them and you don’t even need to ask in a 2PC jurisdiction, it’s already taken care of, just record the call.
- araes 3y ago
  Which implies most humans calling any form of customer service should probably have a local audio recording, because otherwise you're in a lopsided witness situation. Don't want there to be a recording? That call just happened to not be recorded. Notably, this is getting difficult anyway with how easy it is to manufacture vocal data.