16 ms·

Insurers launch cover for losses caused by AI chatbot errors

136 points by jmacd 1y ago

Neywiny 1y ago
No mercy. Had to deal with one when looking for apartments and it made up whatever it thought I wanted to be right. Good thing they still had humans around in person when I went for a tour.
tomrod 1y ago
https://archive.is/BrLso https://archive.is/BrLso
conartist6 1y ago
Man I wish I could get insurance like that. "Accountability insurance" You were responsibile for something, say, child care, and you just decided to go for beer and leave the child with an AI. The house burns down, but because you had insurance you are not responsible. You just head along to your next child care job and don't too much worry about it.
- WrongAssumption 1y ago
  Being covered does not mean you are not responsible.
  - conartist6 1y ago
    That was basically my whole point. Would you want to insure people who think they have no responsibility because they've delegated it to an AI? They might as well have delegated the responsibility to a child or a dog. To sell them insurance, you as the insurer are making a financial bet on the ability of the dog to take care of anything that does go wrong. And still as the insured, using the AI imbued with your responsibility risks horrible outcomes that could still ruin your life. The AI has no life to ruin. It was never really responsible.
    - wat10000 1y ago
      It's just a numbers game. Set your premiums such that you take in more than you pay out. If losses due to dumb use of AI are common then the premiums will be high, but there's no reason to refuse to issue such policies altogether.
- alexriddle 1y ago
  Lots of insurance covers these types of situation which are the result of careless acts... Don't take the right safety precautions and burn down a customers house - liability insurance Click on a link in a phishing email and open up your network to a ransomware attack - cyber insurance Forget to lock your door and get burgled - property insurance Write buggy software which leads to a hospital having to suspend operations - PI (or E&O) insurance Fail to adequately adhere to regulatory obligations and get sued - D&O insurance Obviously there will be various conditions etc which apply but I've been in Insurance a long time and cover for carelessness and stupidity is one of the things which keeps the industry going. I've dealt directly with (paid) claims for all of the above situations. It doesn't absolve responsibility though, it just protects against the financial loss. I suspect if you leave a child alone with an AI and the house burns down that's going to be the least of your problems.
  - jpc0 1y ago
    > Forget to lock your door and get burgled - property insurance I’m pretty sure this will be the same for the other insurance you mentioned but for property insurance if you left your front door open you will have a hard time getting the insurance to actually pay out your claim. At least here they require a burglar alarm and they require it to be armed when nobody is on site or they will absolutely decline the claim. Insurance insures against risk, but there’s a threshold to that and if you prove to be above it they will decline your claim or void your insurance in totality.
    - alexriddle 1y ago
      In the UK where I am, most standard (not budget) property policies would cover theft from an unlocked entry point. Two main exceptions: 1 - if you are letting the property to someone else, e.g a lodger or have paying guests staying with you then this is typically excluded. 2 - if you have had previous theft claims, live in a high crime area, or you have a particularly high risk (e.g lots of valuables), the Insurer will add an endorsement that you need a minimum standard of locks and have them engaged when the property is unoccupied. Outside of those, if you accidentally leave a door unlocked, your claim will likely be paid. The situation obviously may be different in other countries. I worked for a property insurer and saw hundreds of these claims (entry via an unlocked entry point) paid during my time there - I also saw many declined because of the above. I suspect that over time the number of policies in the 'budget' category will continue to increase as price continues to trump everything else for most people] edit: it is the same for the other lines I mentioned as well -e.g a cyber policy I saw recently has no conditions relating to use of MFA. It will have been factored in when writing the risk (they will have said they use it) and if it turned out it was a lie then there would be an issue with cover but if it was just a case of an admin forgetting to include an OU in the MFA group policy the claim would almost certainly be covered. Policies aimed at the SME space are much more likely to have specific conditions though.
- kube-system 1y ago
  Insurance can’t go to jail for you but it can and often does pay your legal fees and/or civil liabilities regardless of fault.
  - tedivm 1y ago
    Yup, I have an umbrella policy to cover a variety of legal situations. It costs me $900 a year for a $3m (per incident) policy.
- Justin_K 1y ago
  It's called errors and omissions and it's as basic an insurance as it gets.
- caulkboots 1y ago
  Not sure insurance will take the rap for criminal negligence.
- thallium205 1y ago
  Crime Insurance (Criminal Acts) is exactly what this is for - when an employee does something criminal while on the clock and the company is facing liability as a result of their actions.
- 0xDEAFBEAD 1y ago
  >You just head along to your next child care job and don't too much worry about it. Aside from the fact that your insurance rate just went up, possibly by a lot.
- john-h-k 1y ago
  This feels like a pretty far fetched straw man. If someone invented medical malpractice insurance yesterday you could use the exact same argument. More generally I think “if something is bad, we should not be able to insure it because then we incentivise it” is not right
- delfinom 1y ago
  Insurance doesn't mean you are not responsible my dude, way to completely misunderstand insurance. Insurance just covers financial damage, and it's the insurer making a bet with you that they will profit off the premiums they calculated for your particular coverage instead of you causing an insurance payout that would be in the red for them. And if you intentionally committed an act that would cause a payout, the insurance would almost certainly void your coverage and claim.
- Suppafly 1y ago
  >Man I wish I could get insurance like that. "Accountability insurance" You could. Insurance companies will sell you insurance for just about anything, in custom situations they figure up the risk somehow. You likely wouldn't like how much it'd cost you though.
loeber 1y ago
Insurance tech guy here. This is not the revolutionary new type of insurance that it might look like at first glance. It's an adaptation of already-commonplace insurance products that are limited in their market size. If you're curious about this topic, I've written about it at length: https://loeber.substack.com/p/24-insurance-for-ai-easier-said-than https://loeber.substack.com/p/24-insurance-for-ai-easier-sai...
- omoikane 1y ago
  Was it also commonplace to have insurances covering human errors? For example: > A tribunal last year ordered Air Canada to honour a discount that its customer service chatbot had made up. If a human sales representative had made that mistake instead of a chatbot, I wonder if companies will try to recover that cost through insurance. Or perhaps AI insurance won't cover the chatbot for that either?
  - loeber 1y ago
    Yes, this is called Professional Liability or Errors & Omissions insurance. It's an important insurance category, but limited in market size. It's uncommon to have e.g. human sales representatives covered for this, but your doctor, lawyer, accountant, architect, etc. will all carry this kind of insurance.
    - kayodelycaon 1y ago
      I worked in this market for a few years. It was fascinating. I still have some ACORD documentation from that. I learned very quickly that standards aren’t. :)
    - kjs3 1y ago
      I carried E&O for years as an independent consultant. I fortunately never had to use it, but I have peers whose financial future was probably saved by having it.
      - SoftTalker 1y ago
imoverclocked 1y ago
At best, this screams, “you’re doing it wrong.” We know this stuff isn’t ready, is easily hacked, is undesirable by consumers… and will fail. Somehow, it’s still more efficient to cover losses and degrade service than to approach the problem differently.
- nickff 1y ago
  Customer service personnel are expensive to train properly, and often quit very quickly because they are treated very poorly by customers. The alternative to AI customer service is often no customer service (like Google).
- rchaud 1y ago
  That assumes that insurers will readily pay out when such claims are made. Insurers don't make money doing that.
- john-h-k 1y ago
  > At best, this screams, “you’re doing it wrong.” If you’re doing it wrong to a meaningful extent you won’t be able to get insurance or it will be very expensive
DonHopkins 1y ago
Can consumers get AI insurance that covers eating a pizza with glue on it, or eating a rock? https://www.forbes.com/sites/jackkelly/2024/05/31/google-ai-glue-to-pizza-viral-blunders/ https://www.forbes.com/sites/jackkelly/2024/05/31/google-ai-... How about MAGA insurance that covers injecting disinfectant, or eating horse dewormer pills, or voting for tariffs?
- 20after4 1y ago
  I'd think it's the rest of us that need to have MAGA insurance, to cover the cost of therapy after realizing how cruel and stupid voting public actually is. And maybe to cover the increased costs of everything due to tariffs.
  - DonHopkins 1y ago
    I want part of the life insurance payouts that any MAGA extremist's family get when they pwned me by not getting vaccinated then dropped dead! It's all my fault I told them to get vaccinated thus forcing them to pwn me. For all my being pwned, I should get paid for my part in purifying the gene pool and raising average human intelligence by a minuscule fraction of a percentage, as well as the right to smugly say I told you so! Because that was ALL part of the plan. Don't let the coffin lid hit you on the way out. The Conservatives Who’d Rather Die Than Not Own the Libs https://www.theatlantic.com/ideas/archive/2021/09/breitbart-conservatives-john-nolte-vaccine/620189/ https://www.theatlantic.com/ideas/archive/2021/09/breitbart-... >Rarely has so significant a faction in American politics behaved in a way that so directly claims the life of its own supporters. [...] >In Nolte’s account, however, a conspiracy of evil leftist elites are to blame for vaccine skepticism on the right. “I sincerely believe the organized left is doing everything in its power to convince Trump supporters NOT to get the life-saving Trump vaccine,” Nolte writes. They are “putting unvaccinated Trump supporters in an impossible position,” he insists, “where they can either NOT get a life-saving vaccine or CAN feel like cucks caving to the ugliest, smuggest bullies in the world.” [...] >Nolte theorized: >In a country where elections are decided on razor-thin margins, does it not benefit one side if their opponents simply drop dead? If I wanted to use reverse psychology to convince people not to get a life-saving vaccination, I would do exactly what Stern and the left are doing … I would bully and taunt and mock and ridicule you for not getting vaccinated, knowing the human response would be, Hey, fuck you, I’m never getting vaccinated! … >Have you ever thought that maybe the left has us right where they want us? Just stand back for a moment and think about this … Right now, a countless number of Trump supporters believe they are owning the left by refusing to take a life-saving vaccine—a vaccine, by the way, everyone on the left has taken. Oh, and so has Trump.
85392_school 1y ago
Reading the actual article, this seems odd. It only covers cases when the models degrade, but there hasn't been evidence of a LLM pinned to a checkpoint degrading yet.
yieldcrv 1y ago
AI that hallucinates accurately enough times should just carry Errors and Omissions insurance like human contractors do
- hoistbypetard 1y ago
  Who in their right mind would underwrite that? Hallucinations are a necessary part of the process, and there's no way to estimate whether the hallucinations are "accurate enough" or not. It'd be like a reverse lottery ticket for the insurance company.
JumpCrisscross 1y ago
Oooh, the foundation-model developers could offer to take first losses up to X if developers follow a rule set. This would reduce premiums and thus increase uptake among users of their models.
fsfod 1y ago
I wonder if the premiums scale up depending on the temperature used for the model output.
Zoethink 1y ago
[dead]
vfclists 1y ago
Pretty sure it will wind up like insurance against malware like NotPetya.
AzzyHN 1y ago
I wonder who makes more errors, underpaid & undertrained employees, or AI chatbots.
otabdeveloper4 1y ago
Whew. Somebody finally figured out how to make money off the nu-AI bubble.
aatd86 1y ago
And now with mcp...should make sure to not allow agents access to sensitive capabilities.

136 points by jmacd 1y ago

No mercy. Had to deal with one when looking for apartments and it made up whatever it thought I wanted to be right. Good thing they still had humans around in person when I went for a tour.

tomrod 1y ago

https://archive.is/BrLso https://archive.is/BrLso

conartist6 1y ago

Man I wish I could get insurance like that. "Accountability insurance" You were responsibile for something, say, child care, and you just decided to go for beer and leave the child with an AI. The house burns down, but because you had insurance you are not responsible. You just head along to your next child care job and don't too much worry about it.

WrongAssumption 1y ago
Being covered does not mean you are not responsible.
- conartist6 1y ago
  That was basically my whole point. Would you want to insure people who think they have no responsibility because they've delegated it to an AI? They might as well have delegated the responsibility to a child or a dog. To sell them insurance, you as the insurer are making a financial bet on the ability of the dog to take care of anything that does go wrong. And still as the insured, using the AI imbued with your responsibility risks horrible outcomes that could still ruin your life. The AI has no life to ruin. It was never really responsible.
  - wat10000 1y ago
    It's just a numbers game. Set your premiums such that you take in more than you pay out. If losses due to dumb use of AI are common then the premiums will be high, but there's no reason to refuse to issue such policies altogether.
alexriddle 1y ago
Lots of insurance covers these types of situation which are the result of careless acts... Don't take the right safety precautions and burn down a customers house - liability insurance Click on a link in a phishing email and open up your network to a ransomware attack - cyber insurance Forget to lock your door and get burgled - property insurance Write buggy software which leads to a hospital having to suspend operations - PI (or E&O) insurance Fail to adequately adhere to regulatory obligations and get sued - D&O insurance Obviously there will be various conditions etc which apply but I've been in Insurance a long time and cover for carelessness and stupidity is one of the things which keeps the industry going. I've dealt directly with (paid) claims for all of the above situations. It doesn't absolve responsibility though, it just protects against the financial loss. I suspect if you leave a child alone with an AI and the house burns down that's going to be the least of your problems.
- jpc0 1y ago
  > Forget to lock your door and get burgled - property insurance I’m pretty sure this will be the same for the other insurance you mentioned but for property insurance if you left your front door open you will have a hard time getting the insurance to actually pay out your claim. At least here they require a burglar alarm and they require it to be armed when nobody is on site or they will absolutely decline the claim. Insurance insures against risk, but there’s a threshold to that and if you prove to be above it they will decline your claim or void your insurance in totality.
  - alexriddle 1y ago
    In the UK where I am, most standard (not budget) property policies would cover theft from an unlocked entry point. Two main exceptions: 1 - if you are letting the property to someone else, e.g a lodger or have paying guests staying with you then this is typically excluded. 2 - if you have had previous theft claims, live in a high crime area, or you have a particularly high risk (e.g lots of valuables), the Insurer will add an endorsement that you need a minimum standard of locks and have them engaged when the property is unoccupied. Outside of those, if you accidentally leave a door unlocked, your claim will likely be paid. The situation obviously may be different in other countries. I worked for a property insurer and saw hundreds of these claims (entry via an unlocked entry point) paid during my time there - I also saw many declined because of the above. I suspect that over time the number of policies in the 'budget' category will continue to increase as price continues to trump everything else for most people] edit: it is the same for the other lines I mentioned as well -e.g a cyber policy I saw recently has no conditions relating to use of MFA. It will have been factored in when writing the risk (they will have said they use it) and if it turned out it was a lie then there would be an issue with cover but if it was just a case of an admin forgetting to include an OU in the MFA group policy the claim would almost certainly be covered. Policies aimed at the SME space are much more likely to have specific conditions though.
kube-system 1y ago
Insurance can’t go to jail for you but it can and often does pay your legal fees and/or civil liabilities regardless of fault.
- tedivm 1y ago
  Yup, I have an umbrella policy to cover a variety of legal situations. It costs me $900 a year for a $3m (per incident) policy.
Justin_K 1y ago
It's called errors and omissions and it's as basic an insurance as it gets.
caulkboots 1y ago
Not sure insurance will take the rap for criminal negligence.
thallium205 1y ago
Crime Insurance (Criminal Acts) is exactly what this is for - when an employee does something criminal while on the clock and the company is facing liability as a result of their actions.
0xDEAFBEAD 1y ago
>You just head along to your next child care job and don't too much worry about it. Aside from the fact that your insurance rate just went up, possibly by a lot.
john-h-k 1y ago
This feels like a pretty far fetched straw man. If someone invented medical malpractice insurance yesterday you could use the exact same argument. More generally I think “if something is bad, we should not be able to insure it because then we incentivise it” is not right
delfinom 1y ago
Insurance doesn't mean you are not responsible my dude, way to completely misunderstand insurance. Insurance just covers financial damage, and it's the insurer making a bet with you that they will profit off the premiums they calculated for your particular coverage instead of you causing an insurance payout that would be in the red for them. And if you intentionally committed an act that would cause a payout, the insurance would almost certainly void your coverage and claim.
Suppafly 1y ago
>Man I wish I could get insurance like that. "Accountability insurance" You could. Insurance companies will sell you insurance for just about anything, in custom situations they figure up the risk somehow. You likely wouldn't like how much it'd cost you though.

loeber 1y ago

Insurance tech guy here. This is not the revolutionary new type of insurance that it might look like at first glance. It's an adaptation of already-commonplace insurance products that are limited in their market size. If you're curious about this topic, I've written about it at length: https://loeber.substack.com/p/24-insurance-for-ai-easier-said-than https://loeber.substack.com/p/24-insurance-for-ai-easier-sai...

omoikane 1y ago
Was it also commonplace to have insurances covering human errors? For example: > A tribunal last year ordered Air Canada to honour a discount that its customer service chatbot had made up. If a human sales representative had made that mistake instead of a chatbot, I wonder if companies will try to recover that cost through insurance. Or perhaps AI insurance won't cover the chatbot for that either?
- loeber 1y ago
  Yes, this is called Professional Liability or Errors & Omissions insurance. It's an important insurance category, but limited in market size. It's uncommon to have e.g. human sales representatives covered for this, but your doctor, lawyer, accountant, architect, etc. will all carry this kind of insurance.
  - kayodelycaon 1y ago
    I worked in this market for a few years. It was fascinating. I still have some ACORD documentation from that. I learned very quickly that standards aren’t. :)
  - kjs3 1y ago
    I carried E&O for years as an independent consultant. I fortunately never had to use it, but I have peers whose financial future was probably saved by having it.
    - SoftTalker 1y ago

imoverclocked 1y ago

At best, this screams, “you’re doing it wrong.” We know this stuff isn’t ready, is easily hacked, is undesirable by consumers… and will fail. Somehow, it’s still more efficient to cover losses and degrade service than to approach the problem differently.

nickff 1y ago
Customer service personnel are expensive to train properly, and often quit very quickly because they are treated very poorly by customers. The alternative to AI customer service is often no customer service (like Google).
rchaud 1y ago
That assumes that insurers will readily pay out when such claims are made. Insurers don't make money doing that.
john-h-k 1y ago
> At best, this screams, “you’re doing it wrong.” If you’re doing it wrong to a meaningful extent you won’t be able to get insurance or it will be very expensive

DonHopkins 1y ago

Can consumers get AI insurance that covers eating a pizza with glue on it, or eating a rock? https://www.forbes.com/sites/jackkelly/2024/05/31/google-ai-glue-to-pizza-viral-blunders/ https://www.forbes.com/sites/jackkelly/2024/05/31/google-ai-... How about MAGA insurance that covers injecting disinfectant, or eating horse dewormer pills, or voting for tariffs?

20after4 1y ago
I'd think it's the rest of us that need to have MAGA insurance, to cover the cost of therapy after realizing how cruel and stupid voting public actually is. And maybe to cover the increased costs of everything due to tariffs.
- DonHopkins 1y ago
  I want part of the life insurance payouts that any MAGA extremist's family get when they pwned me by not getting vaccinated then dropped dead! It's all my fault I told them to get vaccinated thus forcing them to pwn me. For all my being pwned, I should get paid for my part in purifying the gene pool and raising average human intelligence by a minuscule fraction of a percentage, as well as the right to smugly say I told you so! Because that was ALL part of the plan. Don't let the coffin lid hit you on the way out. The Conservatives Who’d Rather Die Than Not Own the Libs https://www.theatlantic.com/ideas/archive/2021/09/breitbart-conservatives-john-nolte-vaccine/620189/ https://www.theatlantic.com/ideas/archive/2021/09/breitbart-... >Rarely has so significant a faction in American politics behaved in a way that so directly claims the life of its own supporters. [...] >In Nolte’s account, however, a conspiracy of evil leftist elites are to blame for vaccine skepticism on the right. “I sincerely believe the organized left is doing everything in its power to convince Trump supporters NOT to get the life-saving Trump vaccine,” Nolte writes. They are “putting unvaccinated Trump supporters in an impossible position,” he insists, “where they can either NOT get a life-saving vaccine or CAN feel like cucks caving to the ugliest, smuggest bullies in the world.” [...] >Nolte theorized: >In a country where elections are decided on razor-thin margins, does it not benefit one side if their opponents simply drop dead? If I wanted to use reverse psychology to convince people not to get a life-saving vaccination, I would do exactly what Stern and the left are doing … I would bully and taunt and mock and ridicule you for not getting vaccinated, knowing the human response would be, Hey, fuck you, I’m never getting vaccinated! … >Have you ever thought that maybe the left has us right where they want us? Just stand back for a moment and think about this … Right now, a countless number of Trump supporters believe they are owning the left by refusing to take a life-saving vaccine—a vaccine, by the way, everyone on the left has taken. Oh, and so has Trump.

85392_school 1y ago

Reading the actual article, this seems odd. It only covers cases when the models degrade, but there hasn't been evidence of a LLM pinned to a checkpoint degrading yet.

yieldcrv 1y ago

AI that hallucinates accurately enough times should just carry Errors and Omissions insurance like human contractors do

hoistbypetard 1y ago
Who in their right mind would underwrite that? Hallucinations are a necessary part of the process, and there's no way to estimate whether the hallucinations are "accurate enough" or not. It'd be like a reverse lottery ticket for the insurance company.

JumpCrisscross 1y ago

Oooh, the foundation-model developers could offer to take first losses up to X if developers follow a rule set. This would reduce premiums and thus increase uptake among users of their models.

fsfod 1y ago

I wonder if the premiums scale up depending on the temperature used for the model output.

Zoethink 1y ago

[dead]

vfclists 1y ago

Pretty sure it will wind up like insurance against malware like NotPetya.

AzzyHN 1y ago

I wonder who makes more errors, underpaid & undertrained employees, or AI chatbots.

otabdeveloper4 1y ago

Whew. Somebody finally figured out how to make money off the nu-AI bubble.

aatd86 1y ago

And now with mcp...should make sure to not allow agents access to sensitive capabilities.