4 ms·

I've worked on chatbots for a couple of years. I too have come to the conclusion that they're generally garbage from a customer perspective. Sometimes they can

I've worked on chatbots for a couple of years. I too have come to the conclusion that they're generally garbage from a customer perspective. Sometimes they can serve as a useful intake system for collecting information before passing someone off to an agent, but even then they tend to be less useful than just filling in a form. I think in most cases, a step by step form followed by a live chat would be better. Unfortunately there is a segment of the population that is completely incapable of filling in a form. These people should probably be passed off to human beings, but then those human beings are wasting their time dealing with similar issues constantly. LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base. I would say a pet peeve of mine is when you do end up having support staff that are human beings, but are not empowered/trained to actually solve customer problems sufficiently. From a business perspective a chatbot is a great replacement for that, because it also regurgitates a script without actually having to action any business processes.

Swizec 2y ago
> LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base. The problem of course is that users have all been trained that knowledge bases are terrible and will go straight to support without even bothering to look. Many users also don’t really want to become an expert in your thing. They just want someone to solve their problem and don’t care how For example: I could learn the ins and outs of quickbooks, but I don’t really wanna. The once or twice per year that the UI won’t let me easily do what I want – straight to support.
burningChrome 2y ago
>> Unfortunately there is a segment of the population that is completely incapable of filling in a form. I work in accessibility. Most of the forms I've worked with are horrendously bad for accessibility. Chat bots are even worse. People with vision or cognitive issues are an afterthought these days. As we've forced more and more people to use these poorly coded or poorly designed applications and moved away from allowing people to talk to an actual person, we are creating an internet that is increasingly less accessible to people who really need it. The sad thing is way too many devs I work with have no idea how to write semantic HTML any more. Just simple things like putting things in an unordered list seems beyond some of the frameworks and CMS they use.
- sublinear 2y ago
  Stop blaming developers and start blaming your organization for not taking accessibility seriously enough. It takes a ton of review, SQA, and dev time to get this stuff right. Writing semantic HTML is just the tip of the iceberg. Nobody is an expert on it either no matter how well they've read WCAG and how much experience they have implementing it. It's a moving target and every page or app is different especially when there are any design or functionality changes. It's a team effort. You can't just dump all responsibility on devs.
- noodlesUK 2y ago
  I agree. I think that a website using (styled) normal semantic HTML forms would be wayyy better for accessibility than some chatbot service which is likely doing some weird things to the markup as a response to stuff coming in on web sockets etc. I think Gov.UK is a great example of how those forms should look most of the time. I would be a strong advocate for some regulatory framework requiring an effective right to access human support for businesses exceeding a certain size.
LeifCarrotson 2y ago
> Unfortunately there is a segment of the population that is completely incapable of filling in a form. These people should probably be passed off to human beings, but then those human beings are wasting their time dealing with similar issues constantly. > LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base. Just as there is a segment of the population that is incapable of filling in a form, there is a (probably larger) segment which is incapable or at least unwilling to educate themselves with a good knowledge base. There are also a lot of bad knowledgebases with incomplete, obsolete, or misleading information out there, which doesn't help the cause.
Karellen 2y ago
> It would be much better to just publish a good knowledge base. Yes, but a lot of companies seem to institutionally incapable of doing so. The number of FAQ pages (or worse, FAQ apps) I've come across filled with questions that no human has ever asked about a product, but instead list all the questions that the marketing department wished people would ask about their product because the answers paint it in such a good light, is astonishing. The trouble with an actual knowledgebase that helps people who are struggling to use your product, is that it broadcasts to the world the ways in which your product is hard to use. And that just doesn't look good. So that info is either not published at all, or made deliberately hard to find, in order that people can't figure out where the warts are.
bombcar 2y ago
> Sometimes they can serve as a useful intake system for collecting information before passing someone off to an agent, but even then they tend to be less useful than just filling in a form. I think in most cases, a step by step form followed by a live chat would be better. A phenomenal amount seem to collect useful identifying information, and then the agent ... asks you for the same information again? I don't know why.
TeMPOraL 2y ago
LLMs could be a great benefit to everyone by making chatbots not shit, if they were used for what they excel at - parsing unstructured natural language inputs, and mapping them to a finite set of (parameterized) options. You could skip the generative part for the bot reply entirely - having a chatbot that understands what I want would be like night and day, even if all it does later is telling me they can't help me.
- flopriore 2y ago
  > You could skip the generative part for the bot reply entirely How could you remove the generative part from an LLM? (genuine question)
  - TeMPOraL 2y ago
    Sorry, bad phrasing. What I mean is, LLMs are more useful when they're asked to parse user input, into a more structured and narrow output, that is then fed to some business logic. Like, there's countless different ways the user could ask about their missing shipment; unlike earlier methods, LLMs are able to correctly classify almost all of them as "missing shipment". So do this, and then stop at that. The conversational output you send to the user does not need to be generated by an LLM. There's not much value added for the massive risk of hallucinations LLMs introduce.

Swizec 2y ago

> LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base. The problem of course is that users have all been trained that knowledge bases are terrible and will go straight to support without even bothering to look. Many users also don’t really want to become an expert in your thing. They just want someone to solve their problem and don’t care how For example: I could learn the ins and outs of quickbooks, but I don’t really wanna. The once or twice per year that the UI won’t let me easily do what I want – straight to support.

burningChrome 2y ago

>> Unfortunately there is a segment of the population that is completely incapable of filling in a form. I work in accessibility. Most of the forms I've worked with are horrendously bad for accessibility. Chat bots are even worse. People with vision or cognitive issues are an afterthought these days. As we've forced more and more people to use these poorly coded or poorly designed applications and moved away from allowing people to talk to an actual person, we are creating an internet that is increasingly less accessible to people who really need it. The sad thing is way too many devs I work with have no idea how to write semantic HTML any more. Just simple things like putting things in an unordered list seems beyond some of the frameworks and CMS they use.

sublinear 2y ago
Stop blaming developers and start blaming your organization for not taking accessibility seriously enough. It takes a ton of review, SQA, and dev time to get this stuff right. Writing semantic HTML is just the tip of the iceberg. Nobody is an expert on it either no matter how well they've read WCAG and how much experience they have implementing it. It's a moving target and every page or app is different especially when there are any design or functionality changes. It's a team effort. You can't just dump all responsibility on devs.
noodlesUK 2y ago
I agree. I think that a website using (styled) normal semantic HTML forms would be wayyy better for accessibility than some chatbot service which is likely doing some weird things to the markup as a response to stuff coming in on web sockets etc. I think Gov.UK is a great example of how those forms should look most of the time. I would be a strong advocate for some regulatory framework requiring an effective right to access human support for businesses exceeding a certain size.

LeifCarrotson 2y ago

> Unfortunately there is a segment of the population that is completely incapable of filling in a form. These people should probably be passed off to human beings, but then those human beings are wasting their time dealing with similar issues constantly. > LLM chatbots are even worse. They have great latitude to say convincing and incorrect information, but can rarely actually do anything. It would be much better to just publish a good knowledge base. Just as there is a segment of the population that is incapable of filling in a form, there is a (probably larger) segment which is incapable or at least unwilling to educate themselves with a good knowledge base. There are also a lot of bad knowledgebases with incomplete, obsolete, or misleading information out there, which doesn't help the cause.

Karellen 2y ago

> It would be much better to just publish a good knowledge base. Yes, but a lot of companies seem to institutionally incapable of doing so. The number of FAQ pages (or worse, FAQ apps) I've come across filled with questions that no human has ever asked about a product, but instead list all the questions that the marketing department wished people would ask about their product because the answers paint it in such a good light, is astonishing. The trouble with an actual knowledgebase that helps people who are struggling to use your product, is that it broadcasts to the world the ways in which your product is hard to use. And that just doesn't look good. So that info is either not published at all, or made deliberately hard to find, in order that people can't figure out where the warts are.

bombcar 2y ago

> Sometimes they can serve as a useful intake system for collecting information before passing someone off to an agent, but even then they tend to be less useful than just filling in a form. I think in most cases, a step by step form followed by a live chat would be better. A phenomenal amount seem to collect useful identifying information, and then the agent ... asks you for the same information again? I don't know why.

TeMPOraL 2y ago

LLMs could be a great benefit to everyone by making chatbots not shit, if they were used for what they excel at - parsing unstructured natural language inputs, and mapping them to a finite set of (parameterized) options. You could skip the generative part for the bot reply entirely - having a chatbot that understands what I want would be like night and day, even if all it does later is telling me they can't help me.

flopriore 2y ago
> You could skip the generative part for the bot reply entirely How could you remove the generative part from an LLM? (genuine question)
- TeMPOraL 2y ago
  Sorry, bad phrasing. What I mean is, LLMs are more useful when they're asked to parse user input, into a more structured and narrow output, that is then fed to some business logic. Like, there's countless different ways the user could ask about their missing shipment; unlike earlier methods, LLMs are able to correctly classify almost all of them as "missing shipment". So do this, and then stop at that. The conversational output you send to the user does not need to be generated by an LLM. There's not much value added for the massive risk of hallucinations LLMs introduce.