11 ms·
Air Canada Has to Honor a Refund Policy Its Chatbot Made Up
- danpalmer 3y agoI'm glad to see that cases are starting to be decided about the liability of using AI generated content. This is something the general public should not need to second-guess.
- bitmasher9 3y agoHonestly LLMs aren’t ready for customer service. If I’m talking to a company I need to have a high degree of accuracy. LLMs are less accurate than trained humans.
- danpalmer 3y agoThis is my personal perception, but I think it's important that there is a clear definition of liability so that companies are able to make their own determinations of what is ready and what isn't.
- denton-scratch 3y agoMe: Chatbot, are you speaking for $COMPANY? Are you $COMPANY's agent? Can I take your statements as being $COMPANY's legal position? Chatbot: <waffle> Me: Please put me through to a person that can articulate $COMPANY's legal position. This conversation can serve no more purpose.
- dtjb 3y agoFew front-line agents have deep knowledge about their company's products or services. They trace their finger through some branches on a flowchart then dictate from a knowledgebase.
- pajamanaut 3y agoFlowcharts are reliable
- dtjb 3y agoAgreed, and I think following flowchart-type logic is within today's AI capabilities. This thread is full of people getting inaccurate responses from humans. I think when it comes to accuracy, a well-trained LLM likely beats the status quo of high-churn low-paid employees following a rote diagram. Of course there should always be a way to reach a human, a senior agent with actual knowledge that can be applied in subjective ways to solve more complex problems.
- yodon 3y agoDupe https://news.ycombinator.com/item?id=39378235 https://news.ycombinator.com/item?id=39378235 (400+ comments)
- ChrisArchitect 3y agoPro-tip OP: https://hn.algolia.com/?dateRange=pastWeek&page=0&prefix=false&query=air%20canada&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=pastWeek&page=0&prefix=fal...
- quartz 3y ago> "Air Canada argues it cannot be held liable for information provided by one of its agents, servants, or representatives—including a chatbot," Rivers wrote. "It does not explain why it believes that is the case" or "why the webpage titled 'Bereavement travel' was inherently more trustworthy than its chatbot." This is very reasonable-- AI or not, companies can't expect consumers to know which parts of their digital experience are accurate and which aren't.
- BugsJustFindMe 3y agoForget about digital experiences for a moment. Forget entirely about chatbots. > Air Canada argues it cannot be held liable for information provided by one of its agents, servants, or representatives That includes EMPLOYEES. So they tried to argue that their employees can lie to your face to get you to buy a ticket under false pretense and then refuse to honor the promised terms? That's absolutely fucked.
- sokoloff 3y agoFortunately, in this case, the consumer had enough proof of what happened and the court rightly told Air Canada to get fucked with that argument.
- mschuster91 3y ago> They tried to argue that their employees can lie to your face to get you to buy a ticket under false pretense and then refuse to honor the promised terms. That's fucked. Pretty standard behavior for big companies. Airlines and telcos are the utter worst... you have agent A on the phone on Monday, who promises X to be done by Wednesday. Thursday, you call again, get agent B, who says he doesn't see anything, not even a call log, from you, but of course he apologizes and it will be done by Friday. (Experienced customers of telcos will know that the drama will unfold that way for months... until you're fed up and involve lawyers)
- ToucanLoucan 3y agoI bet if we removed the requirement to get a lawyer and file a lawsuit that those behaviors would vanish real quick, if all you had to do as the wronged consumer was report to an authority that Company X is doing business dishonestly.
- JJMcJ 3y agoIn early days of computerization, companies tried to dodge liability due to "computer errors". That didn't work, and I hope the "It was the AI, not us" never gets allowed either.
- lenerdenator 3y agoThey still do. Bank errors are an example.
- Aardwolf 3y ago> That didn't work It worked in the British Post Office Scandal: https://en.m.wikipedia.org/wiki/British_Post_Office_scandal https://en.m.wikipedia.org/wiki/British_Post_Office_scandal
- hnfong 3y agoSince it's a scandal, it worked until it didn't. And AFAICT "the computer did it" wasn't the argument, it was "the computer did it so it must be correct because the experts said so".
- alexey-salmin 3y agoSo did they held any humans accountable then? This wasn't the case when I checked but I probably missed some updates
- denton-scratch 3y ago> So did they held any humans accountable then? Not yet. The Inquiry is still taking evidence. They haven't taken evidence from the big-hitters yet, that begins in April. In fact the Inquiry doesn't "hold humans accountable"; but they can compel witnesses, who testify under oath. Gerald Barnes is the Fujitsu engineer who stated in the prosecution cases that Horizon was reliable ("robust"). His testimony to the Inquiry has been delayed, because on the morning he was supposed to testify, the Post Office "discovered" a million or so emails that they'd failed to disclose. So he'll be on the stand in April, along with the senior execs. The Inquiry videos make quite pleasant watching; the lawyers and the judge are immaculately polite, there are no trick questions, and it's all about finding out what happened. I'm looking forward to seeing the senior execs on the stand. The police are following the Inquiry; nobody's been charged, and my guess is they'll hold off on charging people until the Inquiry is over (that's part of the purpose of statutory inquiries). So the succession of Post Office Ministers that have overseen this disgrace will all be out of office by then.
- seguemeters31 3y ago[dead]
- hunter2_ 3y agoI don't really understand why generative LLM output is being presented to (and interpreted by) users as intelligence. This seems incredibly disingenuous, and I would prefer to see it characterized as as fiction, creative, realism, etc. -- words that make it clear to average people that while this might be entertaining and even mimicking reality, it's completely distinct from the writing of a person. Disclaimers (often small, low contrast, or "ToS;DR"-esque) are insufficient when the UI is specifically crafted to appear like chatting with a person.
- im3w1l 3y agoI hope there can be some reasonable middle ground. I think in this case it's good the woman got her money. But Air Canada, presumably scared of what the next case might cost them decided to turn the chat bot off entirely. I think that's a bit unfortunate. I don't know what the solution looks like. Maybe some combination of courts only upholding "reasonable" claims by AI. And then insurance to cover the gaps?
- pjc50 3y agoThe "reasonable" middle ground is that chatbots should not be used unless and until their answers are reliable.
- avn2109 3y ago>> "... turn the chat bot off entirely" This sounds like an ideal outcome!
- lgleason 3y agoIf the company is not confident with the output of the chatbot or the liability then they should turn it off. All business decisions are based on assessing risk. Ultimately either insurance will need to step in or the company is willing to take the hit for mistakes their system makes because it saves them more money. This is how fraud protection it assessed with banks, credit cards etc. as well.
- bgirard 3y agoI agree. If bad information is given and the customer makes a decision on it, then either the customer must bear the risk or the business. I think it's more fair for the business to be out the money. In this case this could have just been a $650 'bug bounty' had Air Canada issued a quick refund. A reasonable QA expense to find out that your AI agent is misleading your customers.
- dukeyukey 3y agoI don't see why chatbots can't be kept to the same standard as human staff. If an airline support agent lied to me to sell me tickets, no shit I'd want a refund and compensation! Chatbots should be allowed to be wrong, but the company should be prepared to face the consequences of that.
- matthewfelgate 3y agohahahahaha
- viranchee 3y agohahahhahaha
- jollyoldpirate 3y ago[flagged]
- VHRanger 3y agoWhy are we posting this when it's just rephrasing an Ars Technica article? It's even mentionned at the bottom: "This story originally appeared on Ars Technica." Give the clicks to the original article: https://arstechnica.com/tech-policy/2024/02/air-canada-must-honor-refund-policy-invented-by-airlines-chatbot/ https://arstechnica.com/tech-policy/2024/02/air-canada-must-...
- hoppyhoppy2 3y agoBoth sites are owned by Conde Nast
- emaro 3y agoI don't like Ars Technica because they break reader mode and load articles chunk by chunk – I consider it hostile towards the user and am glad since Wired work much better.
- chucksmash 3y agoWhat browser? Just tried it out on Firefox for Android (version 122.1.0, with uBlock Origin enabled but JS still allowed on ars) and for the link above, I see the whole article after immediately switching to reader mode.
- PontifexMinimus 3y agoReader mode also works for me. (Firefox 122.0 for Ubuntu)
- deleted 3y ago[deleted]
- lgleason 3y agoGood. If you use a tool that does not give the correct answers you should be held liable for the mistake. The takeaway is, you better vet your tool. If the amount of money you loses from mistakes with the tool is less than the money you saved using it then you make money, if not, you may want to reconsider that cost saving measure.
- lenerdenator 3y agoWhat if the company responds that they don't know how to vet the tool? After all, we're still not 100% sure how LLMs make their decisions in what they string together as output, so the company's not _technically_ lying.
- lazycog512 3y agoThis is why you link the source content from the RAG pipeline instead of pretending the bot knows everything.
- anonu 3y agoReminds me of https://www.moralmachine.net/ https://www.moralmachine.net/
- floatrock 3y agoHere's the real punchline: > Experts told the Vancouver Sun that Air Canada may have succeeded in avoiding liability in Moffatt's case if its chatbot had warned customers that the information that the chatbot provided may not be accurate. Here's a glimpse into our Kafka-esque AI-powered future: every corporate lawyer is now making sure any customer service request will be gated by a chatbot containing a disclaimer like "Warning: the information you receive may be incorrect and irrelevant." Getting correct and relevant information from a human will be impossible.
- unsupp0rted 3y agoThat's why I get all my questions answered by CC'ing legal@example.com and privacy@example.com
- mathgradthrow 3y agothe standard disclaimer for fraud
- OnionBlender 3y agoIf that is such a perfect way to avoid getting sued, why don't they put that on every page of their website and train all of their customer service staff to say that when they talk to customers?
- effingwewt 3y ago....Because an actual judge hadn't said it yet. I still.remember when Microsoft updated thr 360 TOS to force arbitration the day after it was deemed legal in a completely separate case. Rest assured there is an incoming flood of TOS updates.
- waveBidder 3y agoDon't know if you've had the pleasure of interacting with US health insurance, but they do this for coverage and cost estimates all the time, so there is unfortunately precedent in the States.
- add-sub-mul-div 3y ago
- upofadown 3y agoPeter Watts comments: * https://www.rifters.com/crawl/?p=10977 https://www.rifters.com/crawl/?p=10977
- jeffbee 3y agoI don't even see how this is a big story. An expressed representation about what a product is or does, how it works, and the terms of sale in consumer contexts are binding on the seller. It is the same in America. If you go into a store and they say that you can return it if you don't like it, then you can. If you buy a TV and the guy at the store tells you it also makes pancakes, you can get your money back if it turns out that it does not make pancakes. This is true even if the representation is made by some 16-year-old kid working at Best Buy. By extension it would still be true even if it is made be an automaton.
- acdha 3y agoThe interest in this story comes from the fact that Air Canada tried to fight it using the argument that they aren’t liable for anything their agents say. Businesses around the world are so eager to start laying people off that it’s almost certain that they will not be the only ones to try a variation on that claim. I see the important point being reinforcing liability laws and especially not allowing them to shirk responsibility with disclaimers — otherwise we’ll just see more companies cut staffing on their phone support to force people to use a system with a “you are responsible for checking everything this chat bot says for accuracy” notice.
- jeffbee 3y agoI mean, they're welcome to try, and this can't be the first attempt. But there's plenty of jurisprudence at least in the USA that you can't disclaim expressed representations.
- dnussbaum 3y agoThis type of failure is becoming more and more common as companies roll out AI systems without robust accuracy audits & human supervision. I'm working on something to make this easier - reach out if I can be helpful (email in bio).
- logicalmonster 3y agoWould a company be liable to uphold its promises if a rogue human customer service agent promised something ridiculous such as 1 million dollars worth of free flights?
- pizzalife 3y agoNo, the specific employee would most likely be liable if there is criminal conduct (this varies obviously). But a chatbot is not a person.
- npongratz 3y ago> But a chatbot is not a person. We live in an interesting world. In the US, a corporation is legally a person, and a chatbot is not a person[0]. I'm looking forward to the first Supreme Court case involving a corporation consisting of chatbots. [0] I'm handwaving in this lead-in to the fantasy here, so, dear reader, please give me a break for oversimplifying and ignoring technicalities.
- bluGill 3y agoThe company would have to prove the human was knowingly acting outside of their job/training and was disciplined for that. Such discipline must be on the path to firing the employee if the behavior isn't corrected. Note that training is important here, an employee who isn't trained is assumed to have more authorization than someone who is. Or in this case they need to take the AI out of service immediately until they can get a corrected version that does not do such a thing. I will accept that the AI can be tricked to do such a thing and remain in service, but only if they can show the tricks are something an honest human wouldn't attempt. (I don't know what this is, but I'll allow the idea for someone else to propose in enough detail that we can debate if a honest people would ever do that)
- grotorea 3y agoGoogle "too good to be true contract law" and there's some info, seems the answer is "no".
- thefatboy 3y ago
- yifanl 3y agohttps://decisions.civilresolutionbc.ca/crt/crtd/en/item/525448/index.do https://decisions.civilresolutionbc.ca/crt/crtd/en/item/5254... The resolution is n amazingly clear piece of legal writing that explains the involved thought process of the the decision and then awarding the damages. I might end up using this pattern for writing out cause and effect.
- lawkwok 3y agoThanks for sharing. I really enjoyed the read and the concise decision-making and legal terms used.
- spywaregorilla 3y agoThe real desire here is to get it to promise a lifetime of free service. edit: arguing that the chatbot is a separate legal entity is a wild claim. It would imply to me that air canada could sue the ai company for damages if it makes bad promises; not that air canada is excused from paying the customer.
- bluGill 3y agoIf I were the agent who made that promise what would they do to me? Fire me - then fire the chatbot. Put me through training - then train the chatbot. Stand behind me and do what I say - then stand behind their chatbot. I bet they already have policies in place for this - while how they apply to AI may be different, they shouldn't let this slide.
- spywaregorilla 3y agoThe key term here is "apparent authority". You don't need to be granted authority for each and every thing you do, but if you start grossly doing things you were not supposed to do, that's mostly just fraud on your end. The common example in textbooks is someone continuing to do business as an employee after having been fired. They can still make valid deals with other entities due to apparent authority if they're not clearly made out to be separated.
- Cpoll 3y ago> promise a lifetime of free service. That might be too wishful thinking. The tribunal would take into account damages, and whether it was reasonable to believe that you're entitled to free service. In this case, the chatbot promised a ~$800 discount, and the tribunal awarded ~$800. But I doubt they'd make the same decision again, or deem the lifetime service enforceable/un-cancellable.
- spywaregorilla 3y agoI think it's within the bounds of reason if you can cite emotional damages to grant pretty much any recourse. It maybe won't fly if you're clearly trying to trick the AI, but I think there will be general plausible deniability. It may feel weird, but it's utterly insane to delegate customer interactions to an agent that has nobody's interest in mind, not even their own, whom you cannot trust to abide by policy. Similarly, if the bot negotiated any sort of special deal, I think it would be very, very difficult to argue that it lacked apparent authority to make deals or that it's not a fair consideration.
- deleted 3y ago[deleted]
- alsetmusic 3y agoMy father died in hospice the night before a flight to see him. I missed the flight because there was no longer any reason to get to the airport before dawn. I called to reschedule a few hours later. The human on the other end rescheduled and gave me a bereavement rate. She told me it was less money, but didn't mention the reason. I didn't put that together until later. She just helped me out because she had compassion. I am too cynical to think that an AI controlled by a corporation will do this.
- liendolucas 3y agoGood. I hope people out there also discover chatbot holes and exploit them. Chatbots are one of the most useless and time wasters things out there, they literally serve for absolutely no purpose. And most of them work exactly like nested dropdowns where you select one option after the other. Oh and when you really want to talk to a human being in almost every scenario that option is not available. What a wonderful world powered by "AI".
- stainablesteel 3y agoi would expect something like this to severely stunt chatbot adoption i think the only reason this should go through is if it didn't have a proper disclaimer at the beginning of the conversation
- frud 3y agoThe real story here is that Air Canada's lawyers argued, among other things, that the chatbot was a separate and independent legal entity from Air Canada and therefore Air Canada was not obligated to honor the made up policy. In other words, this was possibly the first historical argument made in a court that AI's are sentient and not automated chattel.
- martopix 3y agoOr maybe it simply means that it's a service offered by another company
- fnordpiglet 3y agoThis should hold. If AI can remind us of what humane policies are, so be it.
- pier25 3y agoDid Air Canada use a chatGPT for their legal defense? Also: > Air Canada essentially argued that "the chatbot is a separate legal entity that is responsible for its own actions," What does this mean? That the chatbot was provided by a third party hence they are responsible for the content provided? Or that, literally, a chat bot can be considered a legal entity?
- jerf 3y agoI think this article's full import is not being properly processed yet by a lot of people. The stock market is in an absolute AI frenzy. But this article trashes one of the current boom's biggest supposed markets. If AIs can't be put in contact with customers without exposing the company to an expected liability cost greater than the cost of a human customer representative, one of their major supposed use cases is gone, and that means the money for that use case is gone too. There's probably halo effects in a lot of other uses as well. Now, in the medium or long term, I expect there to be AIs that will be able to do this sort of thing just fine. As I like to say I expect future AIs will not "be" LLMs but merely use LLMs as one of their component parts, and the design as a whole will in fact be able to accurately and reliably relay corporate policies as a result. But the stock market is not currently priced based on "AIs will be pretty awesome in 2029", they're priced on "AIs are going to be pretty awesome in July". LLMs are a huge step forward, but they really aren't suitable for a lot of uses people are trying to put them to in the near term. They don't really "know" things, they're really, really good at guessing them. Now, I don't mean this in the somewhat tedious "what is knowing anyhow" sense, I mean that they really don't have any sort of "facts" in them, just really, really good language skills. I fully expect that people are working on this and the problem will be solved in some manner and we will be able to say that there is an AI design that "knows" things. For instance, see this: https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/ https://deepmind.google/discover/blog/alphageometry-an-olymp... That's in the direction of what I'm talking about; this system does not just babble things that "look" or "sound" like geometry proofs, it "knows" it is doing geometry proofs. This is not quite ready to be fed a corporate policy document, but it is in that direction. But that's got some work to be done yet. (And again, I'm really not interested in another rehash of what "knows" really means. In this specific case I'm speaking of the vector from "a language model" and "a language model + something else like a symbolic engine" as described in that post, where I'm simply defining the latter as "knowing" more about geometry than the former.)
- dlqx 3y agoThis mistake could have been done by a human agent as well and the consequences would have been most likely the same, wouldn't it?
- zzz999 3y agoSometimes the courts get it right