6 ms·

The coming collision between autonomous vehicles and the liability system (2012) [pdf]

33 points by tokai 6y ago

krisoft 6y ago
> “ This first category of defect is unlikely to apply very often to autonomous vehicles, since modern manufacturing methods, especially for critical components of autonomous vehicles such as the software and navigation systems, can be manufactured with low error rates.” That is a very strange thing to say. I would think this assertion is not settled at all, and actually the hearth of the matter.
- tremon 6y ago
  The first category is a manufacturing defect, where the product departs from its intended design Errors of this kind are transfer errors, where (for software) the image loaded onto the vehicle is not bit-for-bit identical to the original code. I would agree with the assessment that these errors are rare, since verification of the process is rather easy: all firmware update procedures already have an established process for verifying the written ROM contents. What they're actually saying is that most navigation errors will stem from design flaws, not from bit-flip errors, which seems a defensible assumption to me.
  - nullc 6y ago
    > errors will stem from design flaws, not from bit-flip errors, which seems a defensible assumption to me. Unless the manufacturer decides that automotive grade parts are overpriced and low performance for the price and starts deploying non-automotive/non-industrial grade parts. Fortunately no reputable automaker would do something like that as they'd surely face dire consequences in the market if they did.
    - throwaway0a5e 6y ago
      Automotive grade is consumer grade. There's standards for some things but they're standards of convenience so that an engineer from GM can call up Bosch and just reference a standard rather than spec it out in detail. Some safety critical things and the overall performance of some systems have standards that need to be met per law but there is no special "automotive grade" for pretty much anything that goes into a car.
      - nullc 6y ago
        Most parts I've seen marked as automotive grade have a radically wider temperature range.
- pritovido 6y ago
  In my experience working with critical hardware and software with sensors, I believe this is pretty much settled, because those components are fixed and well known. We know with precision how to maintain those, the rate of failure and audit or overhaul times. In fact, right know sensors could "call home" when there is a problem. The heart of the matter is what is not known, like the wild board getting in the road, or a cyclist that appears from the side, or the road is wet, there is ice or snow on the road. Usually what happens for accidents is all of the above, like it is dark(you don't see cars but specular reflections of lights), it is wet, there is ice and snow and a cyclist appears out of nowhere. Those things are extremely unknown and hard to model, and AI does not work there.
  - noodlesoups 6y ago
    >Those things are extremely unknown and hard to model, and AI does not work there. That might be because they're problems for _any_ thing. Can't really apply a solution to a problem with what hasn't been solved in the first place. No human is doing any better either, so it'd be unfair to expect autonomous vehicles to do miracles either. For some reason it's fine that Bob just killed a family of four, but unfathomable for a thinking rock to make the slightest error.
    - dredds 6y ago
      Bob was having yet another bad day, and everyone knows Bob drives like he owns the road, and is really just a ticking time bomb. Bob doesn't heed any of the caution his colleagues have, nor care. That clever rock had many of the brightest geniuses on the planet working around the clock for decades steadily making it less prone to calamity. We entrust our lives and family on that proverbial magic carpet ride made by unseen digital gods. It's almost a religious level of faith and expectation. They thought rock-et science was hard...
redis_mlc 6y ago
I'm surprised that insurance companies insure Tesla drivers at all, since drivers are relying on a fake autopilot. What will likely happen is that Tesla will have to offer their own insurance. This avoids the whole problem of insurance companies needing to quantify risk pools.
pritovido 6y ago
I don't see the problem if rate of accidents and incidents are way lower, and they will be. If rate of accidents and incidents gets to 10 times less, the price of insurance will drop 10 times. Claims go to manufacturers? Manufacturers will insure, and you will pay that price, that will be smaller than current insurance. We know the rate of accidents could drop enormously because we have seen what has happened with airplanes. I modeled mathematically accident causes for a big insurer in Spain. They have a database of accidents and most accidents are preventable. 10x less accidents is very conservative and easy to get in a small period of time(10 years). Something as simple as "knowing the road conditions", like "this place is dangerous after the bridge in winter because there is shadow and water and ice is created", or "here there are construction's trucks that leave sand at the curve, dangerous for motorbikes", "here children go out the school that is near and cross the road instead of taking the bridge". When a car can download all this information and take decisions based on that, I expect the rate of accidents to drop dramatically.
- roenxi 6y ago
  Makes a lot of sense. Instead of insurance premiums being paid directly by consumers it'd be paid by car manufacturers who pass it on by slight price increases. It could work very well. I'm pretty extreme, I'd even accept a look-the-other-way policy for the first 10 years where vehicle manufacturers are given a few extra shields from liability. The potential for this tech to save thousands of young lives is very real and we should be bringing it to fruition with indecent haste.
  - unishark 6y ago
    Individuals aren't generally the target of class-action lawsuits or giant billion dollar judgements to force a collective behavior change (or just outright bankruptcy) of every driver in one judgement. They could do this once big companies become responsible for a large number of "drivers" at once. I am also hopeful technological progress will be allowed to win out, however.
- CPLX 6y ago
  Do you realize that what you’re describing is essentially a general AI?
spurgu 6y ago
This aspect of autonomous vehicles is very boring compared to the ethical dilemma. Should self-driving cars perhaps have a morality setting, with a range from "Screw everyone else" to "Save as many lives as possible (prioritize younger), don't worry about me at all"? And should this in turn be connected to your insurance policy (the more selfish the setting, the more you pay)? https://en.wikipedia.org/wiki/Trolley_problem#Implications_for_autonomous_vehicles https://en.wikipedia.org/wiki/Trolley_problem#Implications_f...
- jessaustin 6y ago
  Nearly every fatal collision could have been avoided by driving more slowly. Human drivers aren't blamed unless their speeds are notably excessive, but a computer system that basically is just turning a knob could always have just taken a bit more time and saved that life. If any "screw everybody" settings arise, the lawsuits will eliminate them soon enough.
- graeme 6y ago
  How often do you expect trolley problems to actually arise? https://www.smbc-comics.com/comic/trolley-5 https://www.smbc-comics.com/comic/trolley-5
  - oldgradstudent 6y ago
    Every time the car in front of you makes an emergency brake and you have to decide whether to rear end the car in front of you or risk the truck following you to rear end you.
    - graeme 6y ago
      In years of human driving this has literally never come up.
- the8472 6y ago
  The trolley problem is much ado about nothing. Or nerd-sniping for ethicists. The vehicle could be set to maximize harm in case of an accident and still perform better than the average human driver by creating fewer accidents in the first place. So if you want to optimize for human lives then don't worry about the trolley problem, instead consider how you can get sufficiently good autopilots into as many hands as possible as soon as possible.
hkt 6y ago
More and more people are giving up on self driving car tech. If it can't cope with American grid systems, it is going to flounder on more complex street layouts. See, Steve Wozniak has given up on autonomous vehicles: https://www.extremetech.com/extreme/300927-steve-wozniak-no-self-driving-cars-in-my-lifetime https://www.extremetech.com/extreme/300927-steve-wozniak-no-... See also, the magic roundabout in Swindon, UK: https://en.m.wikipedia.org/wiki/Magic_Roundabout_(Swindon) https://en.m.wikipedia.org/wiki/Magic_Roundabout_(Swindon) - I'd love to see a self driving car deal with this. Still, as a philosophical question it has a fairly clear answer. Generally we go for decision-makers as those who are responsible for decisions made. There's a clear line between the vehicles and the manufacturers. The only way I could see it being construed as the drivers responsibility is if people are given control of the risk profile of the vehicle or if they eg install a custom unit to do the driving. Both seem insanely reckless, although I suppose that has never stopped such developments from taking place before..
- jjcon 6y ago
  > More and more people are giving up on self driving car tech What do you mean when you say this? I’m seeing more and more of them on the roads not less.
- awinter-py 6y ago
  magic roundabout is only the fourth scariest junction in britain

33 points by tokai 6y ago

> “ This first category of defect is unlikely to apply very often to autonomous vehicles, since modern manufacturing methods, especially for critical components of autonomous vehicles such as the software and navigation systems, can be manufactured with low error rates.” That is a very strange thing to say. I would think this assertion is not settled at all, and actually the hearth of the matter.

tremon 6y ago
The first category is a manufacturing defect, where the product departs from its intended design Errors of this kind are transfer errors, where (for software) the image loaded onto the vehicle is not bit-for-bit identical to the original code. I would agree with the assessment that these errors are rare, since verification of the process is rather easy: all firmware update procedures already have an established process for verifying the written ROM contents. What they're actually saying is that most navigation errors will stem from design flaws, not from bit-flip errors, which seems a defensible assumption to me.
- nullc 6y ago
  > errors will stem from design flaws, not from bit-flip errors, which seems a defensible assumption to me. Unless the manufacturer decides that automotive grade parts are overpriced and low performance for the price and starts deploying non-automotive/non-industrial grade parts. Fortunately no reputable automaker would do something like that as they'd surely face dire consequences in the market if they did.
  - throwaway0a5e 6y ago
    Automotive grade is consumer grade. There's standards for some things but they're standards of convenience so that an engineer from GM can call up Bosch and just reference a standard rather than spec it out in detail. Some safety critical things and the overall performance of some systems have standards that need to be met per law but there is no special "automotive grade" for pretty much anything that goes into a car.
    - nullc 6y ago
      Most parts I've seen marked as automotive grade have a radically wider temperature range.
pritovido 6y ago
In my experience working with critical hardware and software with sensors, I believe this is pretty much settled, because those components are fixed and well known. We know with precision how to maintain those, the rate of failure and audit or overhaul times. In fact, right know sensors could "call home" when there is a problem. The heart of the matter is what is not known, like the wild board getting in the road, or a cyclist that appears from the side, or the road is wet, there is ice or snow on the road. Usually what happens for accidents is all of the above, like it is dark(you don't see cars but specular reflections of lights), it is wet, there is ice and snow and a cyclist appears out of nowhere. Those things are extremely unknown and hard to model, and AI does not work there.
- noodlesoups 6y ago
  >Those things are extremely unknown and hard to model, and AI does not work there. That might be because they're problems for _any_ thing. Can't really apply a solution to a problem with what hasn't been solved in the first place. No human is doing any better either, so it'd be unfair to expect autonomous vehicles to do miracles either. For some reason it's fine that Bob just killed a family of four, but unfathomable for a thinking rock to make the slightest error.
  - dredds 6y ago
    Bob was having yet another bad day, and everyone knows Bob drives like he owns the road, and is really just a ticking time bomb. Bob doesn't heed any of the caution his colleagues have, nor care. That clever rock had many of the brightest geniuses on the planet working around the clock for decades steadily making it less prone to calamity. We entrust our lives and family on that proverbial magic carpet ride made by unseen digital gods. It's almost a religious level of faith and expectation. They thought rock-et science was hard...

redis_mlc 6y ago

I'm surprised that insurance companies insure Tesla drivers at all, since drivers are relying on a fake autopilot. What will likely happen is that Tesla will have to offer their own insurance. This avoids the whole problem of insurance companies needing to quantify risk pools.

pritovido 6y ago

I don't see the problem if rate of accidents and incidents are way lower, and they will be. If rate of accidents and incidents gets to 10 times less, the price of insurance will drop 10 times. Claims go to manufacturers? Manufacturers will insure, and you will pay that price, that will be smaller than current insurance. We know the rate of accidents could drop enormously because we have seen what has happened with airplanes. I modeled mathematically accident causes for a big insurer in Spain. They have a database of accidents and most accidents are preventable. 10x less accidents is very conservative and easy to get in a small period of time(10 years). Something as simple as "knowing the road conditions", like "this place is dangerous after the bridge in winter because there is shadow and water and ice is created", or "here there are construction's trucks that leave sand at the curve, dangerous for motorbikes", "here children go out the school that is near and cross the road instead of taking the bridge". When a car can download all this information and take decisions based on that, I expect the rate of accidents to drop dramatically.

roenxi 6y ago
Makes a lot of sense. Instead of insurance premiums being paid directly by consumers it'd be paid by car manufacturers who pass it on by slight price increases. It could work very well. I'm pretty extreme, I'd even accept a look-the-other-way policy for the first 10 years where vehicle manufacturers are given a few extra shields from liability. The potential for this tech to save thousands of young lives is very real and we should be bringing it to fruition with indecent haste.
- unishark 6y ago
  Individuals aren't generally the target of class-action lawsuits or giant billion dollar judgements to force a collective behavior change (or just outright bankruptcy) of every driver in one judgement. They could do this once big companies become responsible for a large number of "drivers" at once. I am also hopeful technological progress will be allowed to win out, however.
CPLX 6y ago
Do you realize that what you’re describing is essentially a general AI?

spurgu 6y ago

This aspect of autonomous vehicles is very boring compared to the ethical dilemma. Should self-driving cars perhaps have a morality setting, with a range from "Screw everyone else" to "Save as many lives as possible (prioritize younger), don't worry about me at all"? And should this in turn be connected to your insurance policy (the more selfish the setting, the more you pay)? https://en.wikipedia.org/wiki/Trolley_problem#Implications_for_autonomous_vehicles https://en.wikipedia.org/wiki/Trolley_problem#Implications_f...

jessaustin 6y ago
Nearly every fatal collision could have been avoided by driving more slowly. Human drivers aren't blamed unless their speeds are notably excessive, but a computer system that basically is just turning a knob could always have just taken a bit more time and saved that life. If any "screw everybody" settings arise, the lawsuits will eliminate them soon enough.
graeme 6y ago
How often do you expect trolley problems to actually arise? https://www.smbc-comics.com/comic/trolley-5 https://www.smbc-comics.com/comic/trolley-5
- oldgradstudent 6y ago
  Every time the car in front of you makes an emergency brake and you have to decide whether to rear end the car in front of you or risk the truck following you to rear end you.
  - graeme 6y ago
    In years of human driving this has literally never come up.
the8472 6y ago
The trolley problem is much ado about nothing. Or nerd-sniping for ethicists. The vehicle could be set to maximize harm in case of an accident and still perform better than the average human driver by creating fewer accidents in the first place. So if you want to optimize for human lives then don't worry about the trolley problem, instead consider how you can get sufficiently good autopilots into as many hands as possible as soon as possible.

hkt 6y ago

More and more people are giving up on self driving car tech. If it can't cope with American grid systems, it is going to flounder on more complex street layouts. See, Steve Wozniak has given up on autonomous vehicles: https://www.extremetech.com/extreme/300927-steve-wozniak-no-self-driving-cars-in-my-lifetime https://www.extremetech.com/extreme/300927-steve-wozniak-no-... See also, the magic roundabout in Swindon, UK: https://en.m.wikipedia.org/wiki/Magic_Roundabout_(Swindon) https://en.m.wikipedia.org/wiki/Magic_Roundabout_(Swindon) - I'd love to see a self driving car deal with this. Still, as a philosophical question it has a fairly clear answer. Generally we go for decision-makers as those who are responsible for decisions made. There's a clear line between the vehicles and the manufacturers. The only way I could see it being construed as the drivers responsibility is if people are given control of the risk profile of the vehicle or if they eg install a custom unit to do the driving. Both seem insanely reckless, although I suppose that has never stopped such developments from taking place before..

jjcon 6y ago
> More and more people are giving up on self driving car tech What do you mean when you say this? I’m seeing more and more of them on the roads not less.
awinter-py 6y ago
magic roundabout is only the fourth scariest junction in britain