2 ms·
> an AI program can likely do that, too Hahaha. Yesterday I spent a half hour with Comcast's AI phone chatbot. No matter how I tried, it could not understand "
by WalterBright 2y ago
> an AI program can likely do that, too
Hahaha. Yesterday I spent a half hour with Comcast's AI phone chatbot. No matter how I tried, it could not understand "cancel Hulu subscription." Finally, it connected me with a human who I said "cancel Hulu subscription" to, and he cancelled it. Mission Accomplished! But the AI chatbot was a total failure.
I am preparing my presentation for DConf, and thought I'd try some relevant pictures. I asked it to "draw a square wheel". It drew a round one. I tried many variations, never got a square wheel, although grok insisted it was a square wheel.
I now and then ask AI to write a function for me. The generated code always looks plausible, but there are always catastrophic bugs in them. The function will include tests (hooray!) but does not include answers to check the results against (boo!).
It doesn't take long to realize that AI has no idea what you're asking for.
AI has a loooong way to go.
- ckcheng 2y ago> Comcast's AI phone chatbot. No matter how I tried, it could not understand "cancel Hulu Seems working as intended? I’ve used a bank and Starbucks chatbots for customer service. Both times they seem so rudimentary and bad that it seemed anything could’ve been better, no LLM required. It took so much effort to get thru to a human. I have to believe it was designed that way, to fob off as many as possible.
- WalterBright 2y agoThe thought definitely occurred to me. Afterwards, it asked me to rate the bot from 0..10. I gave it a zero. I don't recall ever having a positive outcome with a customer service chatbot.
- bell-cot 2y agoA couple years ago, at $Job, I got a call from a Comcast "customer satisfaction" surveybot, after a frustrating interaction with Comcast's "press any button to accomplish more nothing" phone system. I gave the latter a very low rating. Our Comcast internet connection went out a couple minutes later.
- glitchc 2y agoSo, working as intended? Without internet access, you can't leave any future negative reviews.
- lurking_swe 2y agothis was likely designed that way. it’s frustrating isn’t it? What i mean is, one could trivially craft an LLM prompt that lists various “automated capabilities” the business has as an array (each one maybe backed by an api) and then ask the LLM to rank the 3 closest matches based on the users query. Then have the user confirm if any of those choices are correct. If not, trigger the “speak to a rep” workflow.
- chii 2y agoit probably was designed that way because the original way of letting the LLM respond directly and make promises led to people asking for 100% discounts, for unreasonable refunds or other weird edge cases that ruined the business!
- spwa4 2y agoYou mean, it was designed that way to make canceling subscriptions as frustrating as possible. Just like they've been doing with foreign call centres.
- WalterBright 2y ago> Just like they've been doing with foreign call centres When a call center tries that with me, I just respond to all their deflections with "cancel my _____". Eventually, they get the message.