11 ms·

Chatbots Don't Deliver Good Customer Service

137 points by rcymerys 9y ago

danieltillett 9y ago
Does anyone know of a single good chatbot? Every single one I have had the misfortune to interact with has been the best advertisement for their owner's competitors that I know of.
ejo3 9y ago
I use probably about 15 cloud SASS products for my company. Over the past year many of them have switched from offering either live chat or email support to chat bots. Often they will send out a marketing email pitching it as a huge improvement in support. My experience with chat bot support ranges from frustrating to infuriating, especially when I need help with something business critical.
- Pigo 9y ago
  I'm not sure what the right adjective is to describe how it makes me feel when I encounter one, especially if it's with a company that we're doing serious business with. Disrespectful? Condescending? I think belittling works, because it definitely gives you the impression that you're concern is trivial. Now that they have your money they just need to placate you with as little effort as possible.
  - ryandrake 9y ago
    I mean, the same is true for phone bots, and companies have been using them as tech support for decades. At least with phone bots, however, there is usually an incantation (often involving pressing zero repeatedly) that can get you to a live human.
throwaway2016a 9y ago
Yet. Chat bots don't deliver good customer service yet. They will get better and eventually get better than humans. It's a new use of technology, give it time to improve. I already know of companies using AI to adapt to customers like this article says chat bots don't do. It's cutting edge but it is out there.
- michaelt 9y ago
  Maybe. But in that time the early adopters' reputations will be in the gutter because chatbots are the worst customer support out there.
  - epicide 9y ago
    I don't think it's a high risk for their reputation, though. Compared to other forms of support, it is still better. I'd rather talk to a chatbot than: - the same (or worse) bot over a phone - someone who only vaguely speaks my language - nothing/nobody at all - someone who speaks my language but spends 10 minutes and 30 "thank you"s to tell me they can't accomplish what I need The only form that wins out is a living person who is well-trained and speaks my language well. Most companies already won't provide that.
    - michaelt 9y ago
      Personally, if a company can't provide support where you speak to a human, I'd prefer that they be honest about that - rather than having my hopes raised then dashed and my time wasted by a chat/phone/e-mail contact that doesn't go to a human.
      - tyingq 9y ago
        I worked on some IVR implementations some time ago. Quite a few would interpret and understand swearing, and route you to a human via a priority call queue. Similar for multiple button presses of the zero key.
  - fragmede 9y ago
    Chatbots aside, Comcast and United Airlines continue to be big consumer businesses despite horrible customer service reputations. Socially it's bad, but economically it's hard to justify spending that much on customer support. (Those companies also enjoy semi-monopoly positions, though, so I wouldn't recommend emulating that unless you enjoy a similar position in the market.)
deegles 9y ago
I develop Alexa skills and I agree with the article. Another point that is missing is that the technology hasn't "closed the loop". We have great ASR and NLU available, but responses still need to be built by hand or templated. This is the major reason bots feel so unnatural. I think there's a lot of opportunity for robust Natural Language Generation solutions to help build voice apps and chat bots that can operate more flexibly. Also, there's a need for a more declarative, high level programming language to describe how you can interact with a bot. Defining them as trees or graphs isn't flexible enough. I've had promising results from experimenting with concepts from Ceptre (https://github.com/chrisamaphone/interactive-lp https://github.com/chrisamaphone/interactive-lp). Fortunately there are a lot of startups experimenting in this space, I think we'll see the most effective practices replicated quickly.
- mannykannot 9y ago
  In my experience, the major reason bots feel unnatural is that they quickly reveal their lack of actual understanding of anything, with the mean time to Turing-test failure being not much longer than the preliminaries (such as getting your account #.) Actually, maybe 'unnatural' is not the quite right word, as I have had some human customer service that is no better in this regard (or perhaps they were actually some of the better chatbots?)
- Cumulonimbus 9y ago
  I would also posit, that the chatbots are not armed with the ability to actually solve problems. I'd use a chatbot, if it could understand that X is a fraudulent transaction, and to have it cancelled. Instead I get , "Would you like to return your item?" -- NO, it's fraud. It seems either the engineers responsible, or the upper management have made it an order to not allow higher functioning bots. Im guessing they're using the bots as yet another layer of obfuscation to get people to go away, just as companies used "longer than expected" delays on phone lines to get people to leave.
- splatcollision 9y ago
  Agree on the need for a declarative high level language. Something akin to Markdown, but for chatbot flows.
pmontra 9y ago
I agree with the conclusion. Voice input instead of text and better understanding and language generation would help. However the current status is not so grim. In a different application domain, a customer of mine uses a chatbot to collect leads of credit requests. Their very basic chatbot converts much better than their form. The difference is so big that it pays for the extra work to manually extract data from those conversations that went to the end but we can't automatically get all the information from (funnily formatted dates, addresses, etc)
romanovcode 9y ago
Wow, what a surprise. I'm only glad that some of them (for example Microsoft) allows you to request a human. First thing to type when you get a chatbot support is "I would like to speak with human". That usually works.
olavgg 9y ago
Chatbots do work, those who say it doesn't work have just tried the really bad ones. Examples of chatbots that do work Nuance's Nina https://www.windstream.com/Support/ https://www.windstream.com/Support/ Jenn (unknown who is behind it) https://www.alaskaair.com/ https://www.alaskaair.com/ Our James https://master.boost.ai/ https://master.boost.ai/
- tyingq 9y ago
  >Jenn (unknown who is behind it) If you meant who built it... http://nextit.com/work/alaska-airlines http://nextit.com/work/alaska-airlines
  - olavgg 9y ago
    Thanks!
- d--b 9y ago
  James: Hi! James: My name is James and I can help you with bank and insurance questions. Write to me in either Norwegian, Swedish, Danish, Finnish or English. Me: Hey, I am French, and am living half in the US and half in France, do I need to purchase insurance in the US? James: We offer a full range of insurances to meet all your needs. James: Select an option: Cars other vehicles Home insurance Travel insurance Health insurance File insurance claim
  - statusgraph 9y ago
    I get the same response when I ask "does my insurance work outside of new york state". It doesn't seem any more intelligent than finding keywords.
    - olavgg 9y ago
      Currently our intents match banking and insurance questions related to the nordic market. This type of question needs training data that is specific to the US region. There is no problem supporting it technically, it is just lack of training data.
ronack 9y ago
These are pretty well-worn arguments against chatbots. There's an overuse of bots for tasks that do not benefit from a conversational interface (checking the weather, surveys, promotions, etc.). But if you stop thinking of bots as sentient beings and instead treat them as domain-specific command line interfaces, you may get more utility out of them. For instance, it's possible just typing/saying "Send me a pizza with sausage and black olives" is the most efficient way to order a pizza.
- jkaptur 9y ago
  I think that the pizza analogy is really useful. We've had intelligent voice-capable agents for pizza ordering for ages: calling the store. People hate it! They want a specialized UI for ordering pizza.
  - Piskvorrr 9y ago
    [citation-needed]
  - mercer 9y ago
    Chat API's usually offer inline keyboards and regular custom keyboards. At least in Telegram you can even replace inline keyboards based on interactions, meaning you could trivially construct a pretty elaborate UI specifically for stuff like ordering pizza's. And even better, the pizza joint can update the conversation with status updates, special deals if you so desire, and you automatically have a history of previous orders that you could simply repeat at a later time. A chat bot strikes me as the perfect UI for these types of things, and at this point, with Messenger, Messages, and Telegram supporting all this, a large number of your users already have 'your' app installed!
    - jkaptur 9y ago
      Right, I think that WeChat basically exemplifies this too. But at that point, is the user really interacting with a "bot", or is the chat platform just an app delivery mechanism like an OS or a web browser? I'm honestly asking, and I think the pizza example is really illuminating - how DO people order somewhat-customizable food via these platforms?
freehunter 9y ago
I run a Facebook page, and they've recently started pushing chats through Messenger every time you post something asking you to spend money to boost the post. Every single time. And if you do boost, they'll beg you to boost again through Messenger. I tried "stop", I tried "cancel", I tried "unsubscribe", and I even tried cursing at it to see if it would stop. It didn't. Eventually I had to go into Messenger settings and block Facebook's chatbot. I reported it as spam, since there's no way to make it stop bothering you. And now I see they're automatically opening Messenger when you visit someone's page to get you to message that business for... I'm not exactly sure what reason.
pavlov 9y ago
Chatbots offer a state-of-the-art user experience brought through a timewarp from the text adventure games of 1981. "You are in a maze of twisty little passages, all alike. There is a CUSTOMER SERVICE REPRESENTATIVE on the ground."
- ysavir 9y ago
  Maybe you're on to something. Perhaps chatbots would be easier to work with if we were able to examine inventory and see a list of available commands?
  - ucarion 9y ago
    It's considered good style to support user messages like "what can I say?" or "where was I?" for precisely this reason.
    - Pica_soO 9y ago
      You are standing in front of a white house.
dyeje 9y ago
Funny to read these comments when not so many months ago chatbots and conversational UI were all the rage.
- freehunter 9y ago
  It still is "all the rage" for a lot of companies who have invested a lot of money to push this UI. What you're seeing here is backlash from people who are forced to use it and found it lacking. The only people who like it are businesses who paid for it and want to force people to use it.
jaclaz 9y ago
Going a little sideways it's not like at any Call Center you usually get to talk (without having first gone through all their standard "script" and possibly become upset and manage to escalate to a higher level of support) with someone that actually can provide you with assistance beyond what is already on the FAQ of the correspondent site. (provided the site FAQ's are actually frequently asked questions and not - as often happens - a bunch of questions that the original website developer jolted down because he/she imagined they will be asked and that noone actually updates with "real" questions asked by customers). So you get to talk with a human, but very often this human is either clueless or cannot really do anything about the issue at hand. (not of course fault of the human, but rather of the way he/she was mis-trained or because of directives coming from the company)
- rm_-rf_slash 9y ago
  Just ask for their supervisor.
  - jaclaz 9y ago
    Yep, that is: >manage to escalate to a higher level of support It's not that easy on many support Call Centers in my experience, not before having gone through the checklist with a number of unuseful suggestions (already tried because listed in FAQ).
- discreditable 9y ago
  Many chatbots manage to be worse than tech support. For a while, my ISP hid support options behind a "chatbot" that was completely useless.
computator 9y ago
> When you're filling a signup form, you instantly know what information they'll need from you. You can decide if you're comfortable sharing them before you engage in filling it. If only that were true! The trend is to ask for a little bit info at a time, then take you to the next page where it asks for other info, and so on. You don't see at a glance the whole form anymore. I resort to filling in fake info on sign-up forms, order pages, employment applications, airline/hotel reservations, etc., just to find out what questions they'll ask so I can decide whether to refuse before I tell them anything. And also to find out key information they tell you only at the end (sales tax, shipping cost, etc.).
- shangxiao 9y ago
  Along similar lines, I detest a trend I've noticed where login forms show the username/email & password on separate forms. It's annoying to have to click through more than is necessary and additionally breaks my password manager's auto login.
  - thanatropism 9y ago
    Tumblr has done this for how long? Worse yet, they have very similar (except for color) buttons for "Log in" and "Sign up". Often if you have bad internet and the last layer of the page hasn't loaded yet, you don't know which button to press.
  - ge0rg 9y ago
    The worst offender here is Microsoft, if you have a personal and a business account under the same email address (that alone is a major UX fuckup). But they add insult to injury by showing you a typical email+password form, and when you press [Tab] after the email address and start typing your password, the form is replaced under your eyes with a dialog to choose which account you want to log in, only then followed by a password prompt.
- KirinDave 9y ago
  They (and we, I admit to doing this with Level) do this because statistically more people make it through the information collection if we stagger it out. People see a fleet of fields and just go. People see 2 fields and they tend to just enter in data. Even if they leave and come back, mobile apps and websites can much mor easily checkpoint this (digression: I think it's easy to checkpoint a form but lots of people don't and only web toolkits make it drop-dead easy to do). Sorry you're on the right hand side of the computer competency power law distribution.
swiley 9y ago
If you can spend the time to write good documentation for your product then you don't need a chatbot, just good search and good formatting. If you can't be bothered to write good documentation then you won't be bothered to feed useful information into the chatbot, which is way more clumsy than a searchable manual anyway.
- gozur88 9y ago
  A great many people won't read your documentation and will become "stuck" until they have a person explain things. If they're customers you pretty much have to tolerate it unless you're willing to lose them as customers.
sebslomski 9y ago
Chatbots are extremely hard. Many of us are jumping on the bot wagon and leverage AI with NLP & intent recognition. But they miss the important bit: It's all about the conversation itself. "If content creation isn't right, the conversation won't delight" (Adrian Zumbrunnen) The technical aspect behind chat bots is only the tip of the iceberg. We need Conversation Experience Designers instead of AI Engineers & UI Designers. Many chatbots start with questions like "how can I help you? Type something". What should I as a customer expect from the bot? When I visit a website, it's pretty soon clear how it works and what to expect. This doesn't apply to bots.
DustinOfDenver 9y ago
Great article! I have been working on my own Chatbot solution based on IBM's Watson (supposedly cutting edge tech)... it is cumbersome... it cannot do something simple like extract a full address from a conversation (I think cities are still in beta)... The good news, there is a definite opportunity for some start up. AI is all the rage and for solving a specific problem with clearly defined parameters (or if you have a team of a few 100 people working on it) it is getting pretty good. But interacting with a human is many things... "defined" is not one of them.
varelse 9y ago
I suspect what's great about assistants like Siri, Alexa, and even Google Now is less that they have voice interfaces and more that they have well-funded support teams behind them to deliver exponentially more efficient routing of queries into appropriate knowledge-bases based on the best of contemporary research driven by their respective corporate thinktanks. Compare and contrast with data mining the logs of call centers into RNNs and hoping for the best. A fun exercise here is to take one's SMS logs and do the same. In my experience, one will achieve an amazing Max Headroomization of one's cliches and trademark expressions, but little else. When confronted with a voice menu system, I used to scream "OPERATOR! OPERATOR! OPERATOR!" at the top of my lungs and bang on the 0 key repeatedly until I got connected to a human. Recently, I've noticed they hang up on this behavior and even force one to wait through the entire braindead list of choices before allowing pro-users to bypass all the inanity because, after all, one must "listen carefully as our options have changed!"
- EGreg 9y ago
  And always recently! LOL
- JadeNB 9y ago
  > Recently, I've noticed they hang up on this behavior I've definitely had them ignore any sensible operator prompts, but actually hanging up? That's a step beyond. Care to name and shame any offenders? (I've always enjoyed the claim, and once or twice believe I've observed the behaviour—though I am sceptical—that a lot of these systems scan for obscenity, and route to an agent. When I get a robocall while I'm out walking (phone for audiobooks), I always curse at it for a while, just in case. Even though it almost never gets an agent, it's a good stress reliever!)
  - michaelmrose 9y ago
    I've worked phone support for a major cell carrier. Among the things I was told there by a manager. - That people that spam zero should be hung up. - We could do a fake accent if we could maintain it through the whole call. - After they had deleted the feature for support to activate your new device as a cost saving measure we were to point users to the proper procedure to do it themselves on the web and tell them that feature was broken despite the fact that leads could still do this. - To make up fake things we had in common to establish a relationship with the customer so they would give us better surveys. Customer service is not their strong point.
thanatropism 9y ago
Siri is not essential. Every time it fails to deliver a feature of my phone, I can thumb my way to it in seconds. So it's actually fun whenever it works. Also: because it's not essential, one gets to adjust to its failures. There's a broad range of queries for which I know Siri will just say "this is what I found online for..."; I tend to just open the browser and use Google's autocomplete instead. So... I have a problem when the chatbot is trying to perform an essential service, like getting a technician to come to my home because cable internet is not working. Imagine medical triage done by chatbots, jesus christ. An MTV chatbot that tries to do interactive music recommendation? This could work. Spotify's algo is already very good, but you can't refine "yeah, I wanted something more like X, but a little like Y. And not X exactly" -- just "give me more like X including lots of X". And nobody gets hurt by a bad song recommendation.
- michaelbuckbee 9y ago
  That's my feeling of Siri usage as well, that it's not reliable enough to just sort of take for granted. But, what's really interesting to me is that Alexa has surpassed that level (at least in my household). So that for routine things, it's not a novelty to use it, but an actual improvement. This is still obviously for non-essential services like music, games, weather, etc. but it's also reliable enough to handle things like turning on/off lights in awkward spots.
  - rhino369 9y ago
    Alexa isn't really a chat bot. You issue commands to her. She does an amazing job at it. Seriously, the voice recognition is truly great. But you are still issuing a command. You've just learned the language she speaks.
sickbeard 9y ago
Trying to have a conversation with Siri is about the same has trying to have one with a chatbot
KirinDave 9y ago
This article is... weird. It basically assumes the Facebook pseudo-menu-driven chatbots people fat-finger into their systems with almost no consideration for UX are the state of the art. It then compares it to a personal agent that actually has a similar model and the worst voice recognition and search precision of the current crop agent software which... isn't even really the same kind of software? What is this article even trying to say? Poorly written bots are bad, well-written agents are good, and voice interfaces aren't as bad as people make them out to be? Cool.
mmartinson 9y ago
I've been spending a good part of a last year working on a chatbot with a very domain-specific scope, with that goal of arriving a product that delivers a modest but effective user experience. It's hard and it takes an unbelievable amount of work to handle myriad of edge cases you get into when a human on the other end tries to treat your bot like a real albeit limited interlocutor in the conversation. The real problem I see is that the last decade of user experience improvements that work well on the web don't really translate into a chat, as an industry we're hopelessly bereft of best practices at this point, and our users notice this and experience it as the frustration of having no idea what to expect from a bot. The title article and comments cite a lot of good examples. NLP and other applications of machine learning will make bots better at delivering correct answers, but making bots feel not-horrible around the edges is about user experience design. Here are some suggestions that have helped me a lot. * Design for failure first Just like mobile-first design gets the brain out of the pattern of tacking on mobile interactions as second class citizens, failure-first design focuses on the primary experience the users have of your bot, it not working. Don't delude yourself into thinking that your NLP intent parsing is going to result in more hits than misses down your happy path to user delight. A human will always sidestep your intended flow by accident, and that human will form judgements about your product based on it's ability to gracefully recover. Luckily the bar is incredibly low here. * Be careful with conversational niceties and over-humanization of tone It's easy to think that friendly banter and emojis can help personify a bot and smooth over the above-mention failure paths, but the novelty of these wear off quickly for a user, and the user is likely to experience more frustration if the conversational tone doesn't match their frustration. It is also extremely easy to end of in the uncanny valley when using friendly conservational copy in the bot messages. Repetition of a robotic message feels benign-if-annoying, but repetition of a cute emoji-laden phrase can feel very off-putting. * Fall back to being a CLI with visibility and helpers If you've ever been stuck working with a bot, you know that all you want is to know what it can do, and how you can get it to do that thing. If you notice the user is in a failure state through keyword matching or repeated failed routing attempts, fall back to a high-visibility list of actions. Having quick-action buttons can make this even smoother. * Train the user on consistent hooks and keywords. When speaking in a human conversation, utterances of 'stop' or 'wait' are almost always respected as context-independent keywords that escape or pause the context of the conversation. If I asked you what you wanted for dinner, and you responded 'stop', I wouldn't try to figure out what kind of food that was. In my project, 'help' 'quit' and 'back' are all respected as keywords, and every context of the conversation implements callbacks to respond in context to each of these. * Ask a lot of questions that are easy to answer Handling raw language is super hard. Routing language into a finite set of options is a lot easier, plus humans feel listened to when asked for clarification or if they have been understood correctly. When taking user input and routing to an action, ask for yes/no confirmation, and provide options like "This is totally wrong". Opportunities like this could be great to collect data about how users are stuck to improve on the experience. It's also validating for the user to feel like they can specify that they were not heard correctly.
Rjevski 9y ago
The issue with customer service is that a lot of companies end up with awful CS because 1) they outsource it to braindead monkeys who hate their job and are paid pennies and 2) there is no way for someone to complete their task without talking to CS (no documentation, etc). So instead of pouring endless money into chatbots, allow people to do whatever they wanted without involving CS. Provide good, up to date and easy to find documentation and a web interface if the task needs data entry (porting your number, changing your billing details, etc). This will reduce the load on your CS which means you can now bring this back in-house and no longer rely on an outsourced disgruntled workforce.
vkou 9y ago
To the surprise of nobody who has ever gone through automated telephone system hell. Whenever I bump into one, I impulsively shout 'zero' 'zero' 'fucking operator' into the phone.
- cdubzzz 9y ago
  I usually just jam the zero key. Most of the time it works. Have you ever used one that emits "typing" sounds in to the phone? AT&T started doing this some time ago on their small business wireless support line. I jam zero until it says, "Ok, enter the number you are calling about", I enter it, and it says, "Ok, let me look that up for you", then keyboard taping sound... I can't even describe my reaction the first time I heard it. So, so stupid.
mercer 9y ago
So, in contrast to the article I personally feel excited about the potential of chatbots - especially as custom and user-friendly CLI's/UI's more than as AI-driven 'human' interfaces, at least for now - in a way I've not felt excited about anything in a long while (both from a business and ideological perspective, but I care more about the latter, all things considered). I'd just like to ask that if anyone is seriously pursuing chatbots, beyond just helping businesses or consultants sell the 'next big thing', I'd really, really love to talk. My email address is on my profile page.
api 9y ago
Chatbots, home IoT, and to some extent VR all strike me as things Silicon Valley wants to be "the next big thing" but are not and likely will not be because few people want them. I absolutely despise chat and voice for any kind of trivial task. I want a UI. Chat and chat bots are annoying. Voice in any form if infuriating. Having to actually call a number on the phone for any reason always sends me into a minor little rage. "Oh, you have to f'ing call for it... sigh." Why would I want more of this kind of experience? Maybe a few people do but I can't see it being a Big Thing(tm). I really do not want an Internet-connected fridge, stove, light switch, or microwave. When I buy such things I look for non-IoT-encrusted versions since those are going to be more reliable and are not going to be conscripted into a botnet or spy on me. IoT features in these kinds of products deliver nothing of value to me. I also really do not want DRMed food. That includes newer Keurigs. There is a healthy vintage Keurig market on eBay for a reason. VR still seems like a fad or a niche product for certain subsets of gamers. There's some market for it but it's no "E-Commerce," "Social," or "Mobile." It's sort of funny to see the SV investment culture producing so many solutions in search of problems after hearing that same culture lecture against this practice for decades. "Do the market research first!" "MVP MVP!" etc. They're not following their own advice.
yebsido1 9y ago
no one can seriously like chatbots
danjc 9y ago
Zork. In 1980. An example of a text interface that worked better then 99% of chat bots today. Chat is not a bolt on, it's a product in its own right.
bryanrasmussen 9y ago
anyone have some statistics/research on how badly chatbots perform in delivering services?
gidim 9y ago
Yep chatbots sucks. The trick for good customer support is to enhance a human agent with AI and not trying to replace it. Suggesting answers to the agent and automatically replying when the confidence is very high.

137 points by rcymerys 9y ago

Does anyone know of a single good chatbot? Every single one I have had the misfortune to interact with has been the best advertisement for their owner's competitors that I know of.

ejo3 9y ago

I use probably about 15 cloud SASS products for my company. Over the past year many of them have switched from offering either live chat or email support to chat bots. Often they will send out a marketing email pitching it as a huge improvement in support. My experience with chat bot support ranges from frustrating to infuriating, especially when I need help with something business critical.

Pigo 9y ago
I'm not sure what the right adjective is to describe how it makes me feel when I encounter one, especially if it's with a company that we're doing serious business with. Disrespectful? Condescending? I think belittling works, because it definitely gives you the impression that you're concern is trivial. Now that they have your money they just need to placate you with as little effort as possible.
- ryandrake 9y ago
  I mean, the same is true for phone bots, and companies have been using them as tech support for decades. At least with phone bots, however, there is usually an incantation (often involving pressing zero repeatedly) that can get you to a live human.

throwaway2016a 9y ago

Yet. Chat bots don't deliver good customer service yet. They will get better and eventually get better than humans. It's a new use of technology, give it time to improve. I already know of companies using AI to adapt to customers like this article says chat bots don't do. It's cutting edge but it is out there.

michaelt 9y ago
Maybe. But in that time the early adopters' reputations will be in the gutter because chatbots are the worst customer support out there.
- epicide 9y ago
  I don't think it's a high risk for their reputation, though. Compared to other forms of support, it is still better. I'd rather talk to a chatbot than: - the same (or worse) bot over a phone - someone who only vaguely speaks my language - nothing/nobody at all - someone who speaks my language but spends 10 minutes and 30 "thank you"s to tell me they can't accomplish what I need The only form that wins out is a living person who is well-trained and speaks my language well. Most companies already won't provide that.
  - michaelt 9y ago
    Personally, if a company can't provide support where you speak to a human, I'd prefer that they be honest about that - rather than having my hopes raised then dashed and my time wasted by a chat/phone/e-mail contact that doesn't go to a human.
    - tyingq 9y ago
      I worked on some IVR implementations some time ago. Quite a few would interpret and understand swearing, and route you to a human via a priority call queue. Similar for multiple button presses of the zero key.
- fragmede 9y ago
  Chatbots aside, Comcast and United Airlines continue to be big consumer businesses despite horrible customer service reputations. Socially it's bad, but economically it's hard to justify spending that much on customer support. (Those companies also enjoy semi-monopoly positions, though, so I wouldn't recommend emulating that unless you enjoy a similar position in the market.)

deegles 9y ago

I develop Alexa skills and I agree with the article. Another point that is missing is that the technology hasn't "closed the loop". We have great ASR and NLU available, but responses still need to be built by hand or templated. This is the major reason bots feel so unnatural. I think there's a lot of opportunity for robust Natural Language Generation solutions to help build voice apps and chat bots that can operate more flexibly. Also, there's a need for a more declarative, high level programming language to describe how you can interact with a bot. Defining them as trees or graphs isn't flexible enough. I've had promising results from experimenting with concepts from Ceptre (https://github.com/chrisamaphone/interactive-lp https://github.com/chrisamaphone/interactive-lp). Fortunately there are a lot of startups experimenting in this space, I think we'll see the most effective practices replicated quickly.

mannykannot 9y ago
In my experience, the major reason bots feel unnatural is that they quickly reveal their lack of actual understanding of anything, with the mean time to Turing-test failure being not much longer than the preliminaries (such as getting your account #.) Actually, maybe 'unnatural' is not the quite right word, as I have had some human customer service that is no better in this regard (or perhaps they were actually some of the better chatbots?)
Cumulonimbus 9y ago
I would also posit, that the chatbots are not armed with the ability to actually solve problems. I'd use a chatbot, if it could understand that X is a fraudulent transaction, and to have it cancelled. Instead I get , "Would you like to return your item?" -- NO, it's fraud. It seems either the engineers responsible, or the upper management have made it an order to not allow higher functioning bots. Im guessing they're using the bots as yet another layer of obfuscation to get people to go away, just as companies used "longer than expected" delays on phone lines to get people to leave.
splatcollision 9y ago
Agree on the need for a declarative high level language. Something akin to Markdown, but for chatbot flows.

pmontra 9y ago

I agree with the conclusion. Voice input instead of text and better understanding and language generation would help. However the current status is not so grim. In a different application domain, a customer of mine uses a chatbot to collect leads of credit requests. Their very basic chatbot converts much better than their form. The difference is so big that it pays for the extra work to manually extract data from those conversations that went to the end but we can't automatically get all the information from (funnily formatted dates, addresses, etc)

romanovcode 9y ago

Wow, what a surprise. I'm only glad that some of them (for example Microsoft) allows you to request a human. First thing to type when you get a chatbot support is "I would like to speak with human". That usually works.

olavgg 9y ago

Chatbots do work, those who say it doesn't work have just tried the really bad ones. Examples of chatbots that do work Nuance's Nina https://www.windstream.com/Support/ https://www.windstream.com/Support/ Jenn (unknown who is behind it) https://www.alaskaair.com/ https://www.alaskaair.com/ Our James https://master.boost.ai/ https://master.boost.ai/

tyingq 9y ago
>Jenn (unknown who is behind it) If you meant who built it... http://nextit.com/work/alaska-airlines http://nextit.com/work/alaska-airlines
- olavgg 9y ago
  Thanks!
d--b 9y ago
James: Hi! James: My name is James and I can help you with bank and insurance questions. Write to me in either Norwegian, Swedish, Danish, Finnish or English. Me: Hey, I am French, and am living half in the US and half in France, do I need to purchase insurance in the US? James: We offer a full range of insurances to meet all your needs. James: Select an option: Cars other vehicles Home insurance Travel insurance Health insurance File insurance claim
- statusgraph 9y ago
  I get the same response when I ask "does my insurance work outside of new york state". It doesn't seem any more intelligent than finding keywords.
  - olavgg 9y ago
    Currently our intents match banking and insurance questions related to the nordic market. This type of question needs training data that is specific to the US region. There is no problem supporting it technically, it is just lack of training data.

ronack 9y ago

These are pretty well-worn arguments against chatbots. There's an overuse of bots for tasks that do not benefit from a conversational interface (checking the weather, surveys, promotions, etc.). But if you stop thinking of bots as sentient beings and instead treat them as domain-specific command line interfaces, you may get more utility out of them. For instance, it's possible just typing/saying "Send me a pizza with sausage and black olives" is the most efficient way to order a pizza.

jkaptur 9y ago
I think that the pizza analogy is really useful. We've had intelligent voice-capable agents for pizza ordering for ages: calling the store. People hate it! They want a specialized UI for ordering pizza.
- Piskvorrr 9y ago
  [citation-needed]
- mercer 9y ago
  Chat API's usually offer inline keyboards and regular custom keyboards. At least in Telegram you can even replace inline keyboards based on interactions, meaning you could trivially construct a pretty elaborate UI specifically for stuff like ordering pizza's. And even better, the pizza joint can update the conversation with status updates, special deals if you so desire, and you automatically have a history of previous orders that you could simply repeat at a later time. A chat bot strikes me as the perfect UI for these types of things, and at this point, with Messenger, Messages, and Telegram supporting all this, a large number of your users already have 'your' app installed!
  - jkaptur 9y ago
    Right, I think that WeChat basically exemplifies this too. But at that point, is the user really interacting with a "bot", or is the chat platform just an app delivery mechanism like an OS or a web browser? I'm honestly asking, and I think the pizza example is really illuminating - how DO people order somewhat-customizable food via these platforms?

freehunter 9y ago

I run a Facebook page, and they've recently started pushing chats through Messenger every time you post something asking you to spend money to boost the post. Every single time. And if you do boost, they'll beg you to boost again through Messenger. I tried "stop", I tried "cancel", I tried "unsubscribe", and I even tried cursing at it to see if it would stop. It didn't. Eventually I had to go into Messenger settings and block Facebook's chatbot. I reported it as spam, since there's no way to make it stop bothering you. And now I see they're automatically opening Messenger when you visit someone's page to get you to message that business for... I'm not exactly sure what reason.

pavlov 9y ago

Chatbots offer a state-of-the-art user experience brought through a timewarp from the text adventure games of 1981. "You are in a maze of twisty little passages, all alike. There is a CUSTOMER SERVICE REPRESENTATIVE on the ground."

ysavir 9y ago
Maybe you're on to something. Perhaps chatbots would be easier to work with if we were able to examine inventory and see a list of available commands?
- ucarion 9y ago
  It's considered good style to support user messages like "what can I say?" or "where was I?" for precisely this reason.
  - Pica_soO 9y ago
    You are standing in front of a white house.

dyeje 9y ago

Funny to read these comments when not so many months ago chatbots and conversational UI were all the rage.

freehunter 9y ago
It still is "all the rage" for a lot of companies who have invested a lot of money to push this UI. What you're seeing here is backlash from people who are forced to use it and found it lacking. The only people who like it are businesses who paid for it and want to force people to use it.

jaclaz 9y ago

Going a little sideways it's not like at any Call Center you usually get to talk (without having first gone through all their standard "script" and possibly become upset and manage to escalate to a higher level of support) with someone that actually can provide you with assistance beyond what is already on the FAQ of the correspondent site. (provided the site FAQ's are actually frequently asked questions and not - as often happens - a bunch of questions that the original website developer jolted down because he/she imagined they will be asked and that noone actually updates with "real" questions asked by customers). So you get to talk with a human, but very often this human is either clueless or cannot really do anything about the issue at hand. (not of course fault of the human, but rather of the way he/she was mis-trained or because of directives coming from the company)

rm_-rf_slash 9y ago
Just ask for their supervisor.
- jaclaz 9y ago
  Yep, that is: >manage to escalate to a higher level of support It's not that easy on many support Call Centers in my experience, not before having gone through the checklist with a number of unuseful suggestions (already tried because listed in FAQ).
discreditable 9y ago
Many chatbots manage to be worse than tech support. For a while, my ISP hid support options behind a "chatbot" that was completely useless.

computator 9y ago

> When you're filling a signup form, you instantly know what information they'll need from you. You can decide if you're comfortable sharing them before you engage in filling it. If only that were true! The trend is to ask for a little bit info at a time, then take you to the next page where it asks for other info, and so on. You don't see at a glance the whole form anymore. I resort to filling in fake info on sign-up forms, order pages, employment applications, airline/hotel reservations, etc., just to find out what questions they'll ask so I can decide whether to refuse before I tell them anything. And also to find out key information they tell you only at the end (sales tax, shipping cost, etc.).

shangxiao 9y ago
Along similar lines, I detest a trend I've noticed where login forms show the username/email & password on separate forms. It's annoying to have to click through more than is necessary and additionally breaks my password manager's auto login.
- thanatropism 9y ago
  Tumblr has done this for how long? Worse yet, they have very similar (except for color) buttons for "Log in" and "Sign up". Often if you have bad internet and the last layer of the page hasn't loaded yet, you don't know which button to press.
- ge0rg 9y ago
  The worst offender here is Microsoft, if you have a personal and a business account under the same email address (that alone is a major UX fuckup). But they add insult to injury by showing you a typical email+password form, and when you press [Tab] after the email address and start typing your password, the form is replaced under your eyes with a dialog to choose which account you want to log in, only then followed by a password prompt.
KirinDave 9y ago
They (and we, I admit to doing this with Level) do this because statistically more people make it through the information collection if we stagger it out. People see a fleet of fields and just go. People see 2 fields and they tend to just enter in data. Even if they leave and come back, mobile apps and websites can much mor easily checkpoint this (digression: I think it's easy to checkpoint a form but lots of people don't and only web toolkits make it drop-dead easy to do). Sorry you're on the right hand side of the computer competency power law distribution.

swiley 9y ago

If you can spend the time to write good documentation for your product then you don't need a chatbot, just good search and good formatting. If you can't be bothered to write good documentation then you won't be bothered to feed useful information into the chatbot, which is way more clumsy than a searchable manual anyway.

gozur88 9y ago
A great many people won't read your documentation and will become "stuck" until they have a person explain things. If they're customers you pretty much have to tolerate it unless you're willing to lose them as customers.

sebslomski 9y ago

Chatbots are extremely hard. Many of us are jumping on the bot wagon and leverage AI with NLP & intent recognition. But they miss the important bit: It's all about the conversation itself. "If content creation isn't right, the conversation won't delight" (Adrian Zumbrunnen) The technical aspect behind chat bots is only the tip of the iceberg. We need Conversation Experience Designers instead of AI Engineers & UI Designers. Many chatbots start with questions like "how can I help you? Type something". What should I as a customer expect from the bot? When I visit a website, it's pretty soon clear how it works and what to expect. This doesn't apply to bots.

DustinOfDenver 9y ago

Great article! I have been working on my own Chatbot solution based on IBM's Watson (supposedly cutting edge tech)... it is cumbersome... it cannot do something simple like extract a full address from a conversation (I think cities are still in beta)... The good news, there is a definite opportunity for some start up. AI is all the rage and for solving a specific problem with clearly defined parameters (or if you have a team of a few 100 people working on it) it is getting pretty good. But interacting with a human is many things... "defined" is not one of them.

varelse 9y ago

I suspect what's great about assistants like Siri, Alexa, and even Google Now is less that they have voice interfaces and more that they have well-funded support teams behind them to deliver exponentially more efficient routing of queries into appropriate knowledge-bases based on the best of contemporary research driven by their respective corporate thinktanks. Compare and contrast with data mining the logs of call centers into RNNs and hoping for the best. A fun exercise here is to take one's SMS logs and do the same. In my experience, one will achieve an amazing Max Headroomization of one's cliches and trademark expressions, but little else. When confronted with a voice menu system, I used to scream "OPERATOR! OPERATOR! OPERATOR!" at the top of my lungs and bang on the 0 key repeatedly until I got connected to a human. Recently, I've noticed they hang up on this behavior and even force one to wait through the entire braindead list of choices before allowing pro-users to bypass all the inanity because, after all, one must "listen carefully as our options have changed!"

EGreg 9y ago
And always recently! LOL
JadeNB 9y ago
> Recently, I've noticed they hang up on this behavior I've definitely had them ignore any sensible operator prompts, but actually hanging up? That's a step beyond. Care to name and shame any offenders? (I've always enjoyed the claim, and once or twice believe I've observed the behaviour—though I am sceptical—that a lot of these systems scan for obscenity, and route to an agent. When I get a robocall while I'm out walking (phone for audiobooks), I always curse at it for a while, just in case. Even though it almost never gets an agent, it's a good stress reliever!)
- michaelmrose 9y ago
  I've worked phone support for a major cell carrier. Among the things I was told there by a manager. - That people that spam zero should be hung up. - We could do a fake accent if we could maintain it through the whole call. - After they had deleted the feature for support to activate your new device as a cost saving measure we were to point users to the proper procedure to do it themselves on the web and tell them that feature was broken despite the fact that leads could still do this. - To make up fake things we had in common to establish a relationship with the customer so they would give us better surveys. Customer service is not their strong point.

thanatropism 9y ago

Siri is not essential. Every time it fails to deliver a feature of my phone, I can thumb my way to it in seconds. So it's actually fun whenever it works. Also: because it's not essential, one gets to adjust to its failures. There's a broad range of queries for which I know Siri will just say "this is what I found online for..."; I tend to just open the browser and use Google's autocomplete instead. So... I have a problem when the chatbot is trying to perform an essential service, like getting a technician to come to my home because cable internet is not working. Imagine medical triage done by chatbots, jesus christ. An MTV chatbot that tries to do interactive music recommendation? This could work. Spotify's algo is already very good, but you can't refine "yeah, I wanted something more like X, but a little like Y. And not X exactly" -- just "give me more like X including lots of X". And nobody gets hurt by a bad song recommendation.

michaelbuckbee 9y ago
That's my feeling of Siri usage as well, that it's not reliable enough to just sort of take for granted. But, what's really interesting to me is that Alexa has surpassed that level (at least in my household). So that for routine things, it's not a novelty to use it, but an actual improvement. This is still obviously for non-essential services like music, games, weather, etc. but it's also reliable enough to handle things like turning on/off lights in awkward spots.
- rhino369 9y ago
  Alexa isn't really a chat bot. You issue commands to her. She does an amazing job at it. Seriously, the voice recognition is truly great. But you are still issuing a command. You've just learned the language she speaks.

sickbeard 9y ago

Trying to have a conversation with Siri is about the same has trying to have one with a chatbot

KirinDave 9y ago

This article is... weird. It basically assumes the Facebook pseudo-menu-driven chatbots people fat-finger into their systems with almost no consideration for UX are the state of the art. It then compares it to a personal agent that actually has a similar model and the worst voice recognition and search precision of the current crop agent software which... isn't even really the same kind of software? What is this article even trying to say? Poorly written bots are bad, well-written agents are good, and voice interfaces aren't as bad as people make them out to be? Cool.

mmartinson 9y ago

I've been spending a good part of a last year working on a chatbot with a very domain-specific scope, with that goal of arriving a product that delivers a modest but effective user experience. It's hard and it takes an unbelievable amount of work to handle myriad of edge cases you get into when a human on the other end tries to treat your bot like a real albeit limited interlocutor in the conversation. The real problem I see is that the last decade of user experience improvements that work well on the web don't really translate into a chat, as an industry we're hopelessly bereft of best practices at this point, and our users notice this and experience it as the frustration of having no idea what to expect from a bot. The title article and comments cite a lot of good examples. NLP and other applications of machine learning will make bots better at delivering correct answers, but making bots feel not-horrible around the edges is about user experience design. Here are some suggestions that have helped me a lot. * Design for failure first Just like mobile-first design gets the brain out of the pattern of tacking on mobile interactions as second class citizens, failure-first design focuses on the primary experience the users have of your bot, it not working. Don't delude yourself into thinking that your NLP intent parsing is going to result in more hits than misses down your happy path to user delight. A human will always sidestep your intended flow by accident, and that human will form judgements about your product based on it's ability to gracefully recover. Luckily the bar is incredibly low here. * Be careful with conversational niceties and over-humanization of tone It's easy to think that friendly banter and emojis can help personify a bot and smooth over the above-mention failure paths, but the novelty of these wear off quickly for a user, and the user is likely to experience more frustration if the conversational tone doesn't match their frustration. It is also extremely easy to end of in the uncanny valley when using friendly conservational copy in the bot messages. Repetition of a robotic message feels benign-if-annoying, but repetition of a cute emoji-laden phrase can feel very off-putting. * Fall back to being a CLI with visibility and helpers If you've ever been stuck working with a bot, you know that all you want is to know what it can do, and how you can get it to do that thing. If you notice the user is in a failure state through keyword matching or repeated failed routing attempts, fall back to a high-visibility list of actions. Having quick-action buttons can make this even smoother. * Train the user on consistent hooks and keywords. When speaking in a human conversation, utterances of 'stop' or 'wait' are almost always respected as context-independent keywords that escape or pause the context of the conversation. If I asked you what you wanted for dinner, and you responded 'stop', I wouldn't try to figure out what kind of food that was. In my project, 'help' 'quit' and 'back' are all respected as keywords, and every context of the conversation implements callbacks to respond in context to each of these. * Ask a lot of questions that are easy to answer Handling raw language is super hard. Routing language into a finite set of options is a lot easier, plus humans feel listened to when asked for clarification or if they have been understood correctly. When taking user input and routing to an action, ask for yes/no confirmation, and provide options like "This is totally wrong". Opportunities like this could be great to collect data about how users are stuck to improve on the experience. It's also validating for the user to feel like they can specify that they were not heard correctly.

Rjevski 9y ago

The issue with customer service is that a lot of companies end up with awful CS because 1) they outsource it to braindead monkeys who hate their job and are paid pennies and 2) there is no way for someone to complete their task without talking to CS (no documentation, etc). So instead of pouring endless money into chatbots, allow people to do whatever they wanted without involving CS. Provide good, up to date and easy to find documentation and a web interface if the task needs data entry (porting your number, changing your billing details, etc). This will reduce the load on your CS which means you can now bring this back in-house and no longer rely on an outsourced disgruntled workforce.

vkou 9y ago

To the surprise of nobody who has ever gone through automated telephone system hell. Whenever I bump into one, I impulsively shout 'zero' 'zero' 'fucking operator' into the phone.

cdubzzz 9y ago
I usually just jam the zero key. Most of the time it works. Have you ever used one that emits "typing" sounds in to the phone? AT&T started doing this some time ago on their small business wireless support line. I jam zero until it says, "Ok, enter the number you are calling about", I enter it, and it says, "Ok, let me look that up for you", then keyboard taping sound... I can't even describe my reaction the first time I heard it. So, so stupid.

mercer 9y ago

So, in contrast to the article I personally feel excited about the potential of chatbots - especially as custom and user-friendly CLI's/UI's more than as AI-driven 'human' interfaces, at least for now - in a way I've not felt excited about anything in a long while (both from a business and ideological perspective, but I care more about the latter, all things considered). I'd just like to ask that if anyone is seriously pursuing chatbots, beyond just helping businesses or consultants sell the 'next big thing', I'd really, really love to talk. My email address is on my profile page.

api 9y ago

Chatbots, home IoT, and to some extent VR all strike me as things Silicon Valley wants to be "the next big thing" but are not and likely will not be because few people want them. I absolutely despise chat and voice for any kind of trivial task. I want a UI. Chat and chat bots are annoying. Voice in any form if infuriating. Having to actually call a number on the phone for any reason always sends me into a minor little rage. "Oh, you have to f'ing call for it... sigh." Why would I want more of this kind of experience? Maybe a few people do but I can't see it being a Big Thing(tm). I really do not want an Internet-connected fridge, stove, light switch, or microwave. When I buy such things I look for non-IoT-encrusted versions since those are going to be more reliable and are not going to be conscripted into a botnet or spy on me. IoT features in these kinds of products deliver nothing of value to me. I also really do not want DRMed food. That includes newer Keurigs. There is a healthy vintage Keurig market on eBay for a reason. VR still seems like a fad or a niche product for certain subsets of gamers. There's some market for it but it's no "E-Commerce," "Social," or "Mobile." It's sort of funny to see the SV investment culture producing so many solutions in search of problems after hearing that same culture lecture against this practice for decades. "Do the market research first!" "MVP MVP!" etc. They're not following their own advice.

yebsido1 9y ago

no one can seriously like chatbots

danjc 9y ago

Zork. In 1980. An example of a text interface that worked better then 99% of chat bots today. Chat is not a bolt on, it's a product in its own right.

bryanrasmussen 9y ago

anyone have some statistics/research on how badly chatbots perform in delivering services?

gidim 9y ago

Yep chatbots sucks. The trick for good customer support is to enhance a human agent with AI and not trying to replace it. Suggesting answers to the agent and automatically replying when the confidence is very high.