9 ms·

> Lots of people are asking how good the self driving has to be before we tolerate it. There’s a simple answer to this. As soon as it’s good enough for Tesla t

> Lots of people are asking how good the self driving has to be before we tolerate it. There’s a simple answer to this. As soon as it’s good enough for Tesla to accept liability for accidents. Until then if Tesla doesn’t trust it, why should I?

genocidicbunny 2y ago
I think this is probably both the most concise and most reasonable take. It doesn't require anyone to define some level of autonomy or argue about specific edge cases of how the self driving system behaves. And it's easy to apply this principle to not only Tesla, but to all companies making self driving cars and similar features.
concordDance 2y ago
Whats the current total liability cost for all Tesla drivers? The average for all USA cars seems to be around $2000/year, so even if FSD was half as dangerous Tesla would still be paying $1000/year equivalent (not sure how big insurance margins are, assuming nominal) per car. Now, if legally the driver could avoid paying insurance for the few times they want/need to drive themselves (e.g. snow? Dunno what FSD supports atm) then it might make sense economically, but otherwise I don't think it would work out.
- Retric 2y ago
  Liability alone isn’t nearly that high. Car insurance payments include people stealing your car, uninsured motorists, rental cars, and other issues not the drivers fault. Further insurance payments also include profits for the insurance company, advertising, billing, and other overhead from running a business. Also, if Tesla was taking on these risks you’d expect your insurance costs to drop.
  - TheCleric 2y ago
    Yeah any automaker doing this would just negotiate a flat rate per car in the US and the insurer would average the danger to make a rate. This would be much cheaper than the average individual’s cost for liability on their insurance.
    - ryandrake 2y ago
      Somehow I doubt those savings would be passed along to the individual car buyer. Surely buying a car insured by the manufacturer would be much more expensive than buying the car plus your own individual insurance, because the car company would want to profit from both.
    - thedougd 2y ago
      And it would be supplementary to the driver’s insurance, only covering incidents that happen while FSD is engaged. Arguably they would self insure and only purchase insurance for Tesla as a back stop to their liability, maybe through a reinsurance market.
    - ywvcbk 2y ago
      What if someone gets killed because of some clear bug/error and the jury decides to award 100s of millions just for that single ? I’m not sure it’s trivial to insurance companies to account for that sort of risk
      - ndsipa_pomu 2y ago
        Not trivial, but that is exactly the kind of thing that successful insurance companies factor into their premiums, or specifically exclude those scenarios (e.g. not covering war zones for house insurance).
      - kalenx 2y ago
        It is trivial and they've done it for ages. It's called reinsurance. Basically (_very_ basically, there's more to it) the insurance company insures itself against large claims.
        ywvcbk 2y ago
        I’m not sure Boeing etc. could have insured any liability risk resulting from engineering/design flaws in their vehicles?
  - concordDance 2y ago
    Good points, thanks.
  - ywvcbk 2y ago
    How much would every death or severe injury caused by FSD cost Tesla? We probably won’t know anytime soon but since unlike anyone else they can afford to pay out virtually unlimited amounts and courts will presumably take that into account
- ywvcbk 2y ago
  Also I wouldn’t be surprised if any potential wrongful death lawsuits could cost Tesla several magnitudes more than the current average.
bdcravens 2y ago
The liability for killing someone can include prison time.
- TheCleric 2y ago
  Good. If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally.
  - beej71 2y ago
    And such coders should carry malpractice insurance.
  - dmix 2y ago
    Drug companies and the FDA (circa 1906) play a very dangerous and delicate dance all the time releasing new drugs to the public. But for over a century now we've managed to figure it out without holding pharma companies criminally liable for every death. > If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally. Easy to type those words on the internet than make it a policy IRL. That sort of policy IRL would likely result in a) killing off all commercial efforts to solve traffic deaths via technology and vast amounts of other semi-autonomous technology like farm equipment or b) government/car companies mandating filming the driver every time they turn it on, because it's technically supposed to be human assisted autopilot in these testing stages (outside restricted pilot programs like Waymo taxis). Those distinctions would matter in a criminal court room, even if humans can't always be relied upon to always follow the instructions on the bottle's label.
    - ryandrake 2y ago
      Your take is understandable and not surprising on a site full of software developers. Somehow, the general software industry has ingrained this pessimistic and fatalistic dogma that says bugs are inevitable and there’s nothing you can do to prevent them. Since everyone believes it, it is a self-fulfilling prophecy and we just accept it as some kind of law of nature. Holding software developers (or their companies) liable for defects would definitely kill off a part of the industry: the very large part that YOLOs code into production and races to get features released without rigorous and exhaustive testing. And why don’t they spend 90% of their time testing and verifying and proving their software has no defects? Because defects are inevitable and they’re not held accountable for them!
tiahura 2y ago
I think that’s implicit in the promise of the upcoming-any-year-now unattended full self driving.
renewiltord 2y ago
This is how I feel about nuclear energy. Every single plant should need to form a full insurance fund dedicated to paying out if there’s trouble. And the plant should have strict liability: anything that happens from materials it releases are its responsibility. But people get upset about this. We need corporations to take responsibility.
- idiotsecant 2y ago
  While we're at it how about why apply the same standard to coal and natural gas plants? For some reason when we start taking about nuclear plants we all of a sudden become adverse to the idea of unfunded externalities but when we're talking about 'old' tech that has been steadily irradiating your community and changing the gas composition of the entire planet it becomes less concerning.
  - moooo99 2y ago
    I think it is a matter of perceived risk. Realistically speaking, nuclear power is pretty safe. In the history of nuclear power, there were two major incidents. Considering the number of nuclear power plants around the planet, that is pretty good. However, as those two accidents demonstrated, the potential fallout of those incidents is pretty severe and widespread. I think this massively contributes to the perceived risks. The warnings towards the public were pretty clear. I remember my mom telling stories from the time the Chernobyl incident became known to the public and people became worried about the produce they usually had from their gardens. Meanwhile, everything that has been done to address the hazards of fossil based power generation is pretty much happening behind the scenes. With coal and natural gas, it seems like people perceive the risks as more abstract. The radioactive emissions of coal power plants have been known for a while and the (potential) dangers of fine particulate matters resulting from combustion are somewhat well known nowadays as well. However, the effects of those danger seem much more abstract and delayed, leading people to not be as worried about it. It also shows on a smaller, more individual scale: people still buy ICE cars at large and install gas stoves into their houses despite induction being readily available and at times even cheaper.
mrjin 2y ago
Even if it does, can it resurrect the deceased?
- LadyCailin 2y ago
  But people driving manually kill people all the time too. The bar for self driving isn’t «does it never kill anyone», it’s «does it kill people less than manual driving». We’re not there yet, and Tesla’s «FSD» is marketing bullshit, but we certainly will be there one day, and at that point, we need to understand what we as a society will do when a self driving car kills someone. It’s not obvious what the best solution is there, and we need to continue to have societal discussions to hash that out, but the correct solution definitely isn’t «don’t use self driving».
  - amelius 2y ago
    No, because every driver thinks they are better than average. So nobody will accept it.
    - the8472 2y ago
      I expect insurance to figure out the relative risks and put a price sticker on that decision.
    - A4ET8a8uTh0 2y ago
      Assuming I understand the argument flow correctly, I think I disagree. If there is one thing that the past few decades have confirmed quite conclusively, it is that people will trade a lot of control and sense away in the name of convenience. The moment FSD reaches that sweet spot of 'take me home -- I am too drunk to drive' of reliability, I think it would be accepted; maybe even required by law. It does not seem there.
    - Dylan16807 2y ago
      The level where someone personally uses it and the level where they accept it being on the road are different. Beating the average driver is all about the latter. Also I will happily use self driving that matches the median driver in safety.
ndsipa_pomu 2y ago
> As soon as it’s good enough for Tesla to accept liability for accidents. That makes a lot of sense and not just from a selfish point of view. When a person drives a vehicle, then the person is held responsible for how the vehicle behaves on the roads, so it's logical that when a machine drives a vehicle that the machine's manufacturer/designer is held responsible. It's a complete con that Tesla is promoting their autonomous driving, but also having their vehicles suddenly switch to non-autonomous driving which they claim moves the responsibility to the human in the driver seat. Presumably, the idea is that the human should have been watching and approving everything that the vehicle has done up to that point.
- andrewaylett 2y ago
  The responsibility doesn't shift, it always lies with the human. One problem is that humans are notoriously poor at maintaining attention when supervising automation Until the car is ready to take over as legal driver, it's foolish to set the human driver up for failure in the way that Tesla (and the humans driving Tesla cars) do.
  - f1shy 2y ago
    What?! So if there is a failure and the car goes full throttle (no autonomous car) it is my responsibility?! You are pretty wrong!!!
    - kgermino 2y ago
      You are responsible (Legally, contractually, morally) for supervising FSD today. If the car decided to stomp on the throttle you are expected to be ready to hit the brakes. The whole point is that is somewhat of an unreasonable expectation but it’s what Tesla expects you to do today
      - f1shy 2y ago
        My example was clear about NOT about autonomous driving. Because the previous comment seems to imply for everything you are responsible
jefftk 2y ago
Note that Mercedes does take liability for accidents with their (very limited level) level 3 system: https://www.theverge.com/2023/9/27/23892154/mercedes-benz-drive-pilot-autonomous-level-3-test https://www.theverge.com/2023/9/27/23892154/mercedes-benz-dr...
- f1shy 2y ago
  Yes. That is the only way. That being said, I want to see the first incidents, and how are they resolved.
- iknowstuff 2y ago
  its pathetic. <40mph following a vehicle directly ahead. basically only usable in stop and go traffic https://www.notebookcheck.net/Tesla-vs-Mercedes-self-driving-test-ends-in-40-interventions-as-Elon-Musk-says-FSD-is-years-ahead.835805.0.html https://www.notebookcheck.net/Tesla-vs-Mercedes-self-driving...
  - jefftk 2y ago
    The Mercedes system is definitely, as I said, very limited. But within it's operating conditions the Mercedes system is much more useful: you can safely and legally read, work, or watch a movie while in the driver's seat, literally not paying any attention to the road.
theptip 2y ago
Presumably that is exactly when their taxi service rolls out? While this has a dramatic rhetorical flourish, I don’t think it’s a good proxy. Even if it was safer, it would be an unnecessarily high burden to clear. You’d be effectively writing a free insurance policy which is obviously not free. Just look at total accidents / deaths per mile driven, it’s the obvious and standard metric for measuring car safety. (You need to be careful to not stop the clock as soon as the system disengages of course. )

genocidicbunny 2y ago

I think this is probably both the most concise and most reasonable take. It doesn't require anyone to define some level of autonomy or argue about specific edge cases of how the self driving system behaves. And it's easy to apply this principle to not only Tesla, but to all companies making self driving cars and similar features.

concordDance 2y ago

Whats the current total liability cost for all Tesla drivers? The average for all USA cars seems to be around $2000/year, so even if FSD was half as dangerous Tesla would still be paying $1000/year equivalent (not sure how big insurance margins are, assuming nominal) per car. Now, if legally the driver could avoid paying insurance for the few times they want/need to drive themselves (e.g. snow? Dunno what FSD supports atm) then it might make sense economically, but otherwise I don't think it would work out.

Retric 2y ago
Liability alone isn’t nearly that high. Car insurance payments include people stealing your car, uninsured motorists, rental cars, and other issues not the drivers fault. Further insurance payments also include profits for the insurance company, advertising, billing, and other overhead from running a business. Also, if Tesla was taking on these risks you’d expect your insurance costs to drop.
- TheCleric 2y ago
  Yeah any automaker doing this would just negotiate a flat rate per car in the US and the insurer would average the danger to make a rate. This would be much cheaper than the average individual’s cost for liability on their insurance.
  - ryandrake 2y ago
    Somehow I doubt those savings would be passed along to the individual car buyer. Surely buying a car insured by the manufacturer would be much more expensive than buying the car plus your own individual insurance, because the car company would want to profit from both.
  - thedougd 2y ago
    And it would be supplementary to the driver’s insurance, only covering incidents that happen while FSD is engaged. Arguably they would self insure and only purchase insurance for Tesla as a back stop to their liability, maybe through a reinsurance market.
  - ywvcbk 2y ago
    What if someone gets killed because of some clear bug/error and the jury decides to award 100s of millions just for that single ? I’m not sure it’s trivial to insurance companies to account for that sort of risk
    - ndsipa_pomu 2y ago
      Not trivial, but that is exactly the kind of thing that successful insurance companies factor into their premiums, or specifically exclude those scenarios (e.g. not covering war zones for house insurance).
    - kalenx 2y ago
      It is trivial and they've done it for ages. It's called reinsurance. Basically (_very_ basically, there's more to it) the insurance company insures itself against large claims.
      - ywvcbk 2y ago
        I’m not sure Boeing etc. could have insured any liability risk resulting from engineering/design flaws in their vehicles?
- concordDance 2y ago
  Good points, thanks.
- ywvcbk 2y ago
  How much would every death or severe injury caused by FSD cost Tesla? We probably won’t know anytime soon but since unlike anyone else they can afford to pay out virtually unlimited amounts and courts will presumably take that into account
ywvcbk 2y ago
Also I wouldn’t be surprised if any potential wrongful death lawsuits could cost Tesla several magnitudes more than the current average.

bdcravens 2y ago

The liability for killing someone can include prison time.

TheCleric 2y ago
Good. If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally.
- beej71 2y ago
  And such coders should carry malpractice insurance.
- dmix 2y ago
  Drug companies and the FDA (circa 1906) play a very dangerous and delicate dance all the time releasing new drugs to the public. But for over a century now we've managed to figure it out without holding pharma companies criminally liable for every death. > If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally. Easy to type those words on the internet than make it a policy IRL. That sort of policy IRL would likely result in a) killing off all commercial efforts to solve traffic deaths via technology and vast amounts of other semi-autonomous technology like farm equipment or b) government/car companies mandating filming the driver every time they turn it on, because it's technically supposed to be human assisted autopilot in these testing stages (outside restricted pilot programs like Waymo taxis). Those distinctions would matter in a criminal court room, even if humans can't always be relied upon to always follow the instructions on the bottle's label.
  - ryandrake 2y ago
    Your take is understandable and not surprising on a site full of software developers. Somehow, the general software industry has ingrained this pessimistic and fatalistic dogma that says bugs are inevitable and there’s nothing you can do to prevent them. Since everyone believes it, it is a self-fulfilling prophecy and we just accept it as some kind of law of nature. Holding software developers (or their companies) liable for defects would definitely kill off a part of the industry: the very large part that YOLOs code into production and races to get features released without rigorous and exhaustive testing. And why don’t they spend 90% of their time testing and verifying and proving their software has no defects? Because defects are inevitable and they’re not held accountable for them!

tiahura 2y ago

I think that’s implicit in the promise of the upcoming-any-year-now unattended full self driving.

renewiltord 2y ago

This is how I feel about nuclear energy. Every single plant should need to form a full insurance fund dedicated to paying out if there’s trouble. And the plant should have strict liability: anything that happens from materials it releases are its responsibility. But people get upset about this. We need corporations to take responsibility.

idiotsecant 2y ago
While we're at it how about why apply the same standard to coal and natural gas plants? For some reason when we start taking about nuclear plants we all of a sudden become adverse to the idea of unfunded externalities but when we're talking about 'old' tech that has been steadily irradiating your community and changing the gas composition of the entire planet it becomes less concerning.
- moooo99 2y ago
  I think it is a matter of perceived risk. Realistically speaking, nuclear power is pretty safe. In the history of nuclear power, there were two major incidents. Considering the number of nuclear power plants around the planet, that is pretty good. However, as those two accidents demonstrated, the potential fallout of those incidents is pretty severe and widespread. I think this massively contributes to the perceived risks. The warnings towards the public were pretty clear. I remember my mom telling stories from the time the Chernobyl incident became known to the public and people became worried about the produce they usually had from their gardens. Meanwhile, everything that has been done to address the hazards of fossil based power generation is pretty much happening behind the scenes. With coal and natural gas, it seems like people perceive the risks as more abstract. The radioactive emissions of coal power plants have been known for a while and the (potential) dangers of fine particulate matters resulting from combustion are somewhat well known nowadays as well. However, the effects of those danger seem much more abstract and delayed, leading people to not be as worried about it. It also shows on a smaller, more individual scale: people still buy ICE cars at large and install gas stoves into their houses despite induction being readily available and at times even cheaper.

mrjin 2y ago

Even if it does, can it resurrect the deceased?

LadyCailin 2y ago
But people driving manually kill people all the time too. The bar for self driving isn’t «does it never kill anyone», it’s «does it kill people less than manual driving». We’re not there yet, and Tesla’s «FSD» is marketing bullshit, but we certainly will be there one day, and at that point, we need to understand what we as a society will do when a self driving car kills someone. It’s not obvious what the best solution is there, and we need to continue to have societal discussions to hash that out, but the correct solution definitely isn’t «don’t use self driving».
- amelius 2y ago
  No, because every driver thinks they are better than average. So nobody will accept it.
  - the8472 2y ago
    I expect insurance to figure out the relative risks and put a price sticker on that decision.
  - A4ET8a8uTh0 2y ago
    Assuming I understand the argument flow correctly, I think I disagree. If there is one thing that the past few decades have confirmed quite conclusively, it is that people will trade a lot of control and sense away in the name of convenience. The moment FSD reaches that sweet spot of 'take me home -- I am too drunk to drive' of reliability, I think it would be accepted; maybe even required by law. It does not seem there.
  - Dylan16807 2y ago
    The level where someone personally uses it and the level where they accept it being on the road are different. Beating the average driver is all about the latter. Also I will happily use self driving that matches the median driver in safety.

ndsipa_pomu 2y ago

> As soon as it’s good enough for Tesla to accept liability for accidents. That makes a lot of sense and not just from a selfish point of view. When a person drives a vehicle, then the person is held responsible for how the vehicle behaves on the roads, so it's logical that when a machine drives a vehicle that the machine's manufacturer/designer is held responsible. It's a complete con that Tesla is promoting their autonomous driving, but also having their vehicles suddenly switch to non-autonomous driving which they claim moves the responsibility to the human in the driver seat. Presumably, the idea is that the human should have been watching and approving everything that the vehicle has done up to that point.

andrewaylett 2y ago
The responsibility doesn't shift, it always lies with the human. One problem is that humans are notoriously poor at maintaining attention when supervising automation Until the car is ready to take over as legal driver, it's foolish to set the human driver up for failure in the way that Tesla (and the humans driving Tesla cars) do.
- f1shy 2y ago
  What?! So if there is a failure and the car goes full throttle (no autonomous car) it is my responsibility?! You are pretty wrong!!!
  - kgermino 2y ago
    You are responsible (Legally, contractually, morally) for supervising FSD today. If the car decided to stomp on the throttle you are expected to be ready to hit the brakes. The whole point is that is somewhat of an unreasonable expectation but it’s what Tesla expects you to do today
    - f1shy 2y ago
      My example was clear about NOT about autonomous driving. Because the previous comment seems to imply for everything you are responsible

jefftk 2y ago

Note that Mercedes does take liability for accidents with their (very limited level) level 3 system: https://www.theverge.com/2023/9/27/23892154/mercedes-benz-drive-pilot-autonomous-level-3-test https://www.theverge.com/2023/9/27/23892154/mercedes-benz-dr...

f1shy 2y ago
Yes. That is the only way. That being said, I want to see the first incidents, and how are they resolved.
iknowstuff 2y ago
its pathetic. <40mph following a vehicle directly ahead. basically only usable in stop and go traffic https://www.notebookcheck.net/Tesla-vs-Mercedes-self-driving-test-ends-in-40-interventions-as-Elon-Musk-says-FSD-is-years-ahead.835805.0.html https://www.notebookcheck.net/Tesla-vs-Mercedes-self-driving...
- jefftk 2y ago
  The Mercedes system is definitely, as I said, very limited. But within it's operating conditions the Mercedes system is much more useful: you can safely and legally read, work, or watch a movie while in the driver's seat, literally not paying any attention to the road.

theptip 2y ago

Presumably that is exactly when their taxi service rolls out? While this has a dramatic rhetorical flourish, I don’t think it’s a good proxy. Even if it was safer, it would be an unnecessarily high burden to clear. You’d be effectively writing a free insurance policy which is obviously not free. Just look at total accidents / deaths per mile driven, it’s the obvious and standard metric for measuring car safety. (You need to be careful to not stop the clock as soon as the system disengages of course. )