19 ms·

If it's a subjective matter tied to perception of risk rather than actual, statistical risk, such perception can be swayed. The challenge remains that people w

If it's a subjective matter tied to perception of risk rather than actual, statistical risk, such perception can be swayed. The challenge remains that people will be killed in accidents involving autonomous control. And we anticipate that the number of people killed will be fewer, hence 'saving lives'. However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. There will be cases where a court determines that the autonomous system was the cause. Families of those killed will want justice, while those separately saved by autonomous systems may never be heard from in the same case. I expect that in the end it will come down to a business decision, and that decision will be informed by an actuarial exercise: Will profits and insurance be able to cover the costs of defending and settling such cases. Who knows, maybe the threshold is crossed at 5x safer.

tgv 6y ago
Idk about risk: that is hard to establish. There are so many conditions in which an automatic pilot hasn't been tested. We don't even know the factors involved in estimating the risk: e.g., is it dependent on the human co-pilot? And self-driving cars may change the car usage patterns, exerting a contextual influence on the risk. Then there's the question of responsibility. Who will be held responsible when the automatic pilot is driving? If it's the human, then a high risk of causing an accident will be unacceptable to many drivers.
Justsignedup 6y ago
Sensationalism will win out quite a bit. And responsibility. Its just a tough problem. - People developing Bell's Palsy at the same rate with vaccine than not. But suddenly only those with the vaccine show up in the social media feeds. Because nobody just posts "I just got bell's palsy" but now they do because people are paying attention. Same will happen with AVs. "I just got into an AV accident" will make headlines, while "I just dozed off and hit a kid" will barely circulate. - People inherently trust humans over technology. Just because. So they will be quick to distrust autonomous vehicles. I already had convos about the fact that yes, teslas do kill, but on the whole self-driving teslas do kill far less than non. - When a human drives, the liability is on the human. When a car self-drives, the liability might be on the manufacturer.
- SkyBelow 6y ago
  >When a human drives, the liability is on the human. When a car self-drives, the liability might be on the manufacturer. I think this is a point that needs more emphasis, especially on the word 'might'. Without both laws and a history of court cases giving evidence for how those laws are interpreted and enforced, it isn't possible to tell where liability may end up. I wonder if that is part of the reason people are hesitant. Liability is being removed from the driver, but it doesn't see have have found a place to settle back down at, so people are viewing it as if liability is just being removed. For initial court cases (and the amount of time and money it takes to fight them), this may not be an unrealistic expectation.
  - ghaff 6y ago
    If liability is on anyone, it would seem it has to be the manufacturer. And if there's no liability then the options are basically set up some sort of vaccine fund-like system or just to shrug and say it's between you and the insurance company.
- cj 6y ago
  > People inherently trust humans over technology. Just because. From the perspective of someone who rides a motorcycle, the #1 thing you need to do to not crash is to anticipate what all vehicles around you might possibly do. For example, I always avoid riding in another car’s blind spot for obvious reasons. The problem (for motorcyclists) will now be trying to adapt to understand what a Tesla might do and where a Tesla’s blind spots might be - and once you add in the idiosyncrasies of other AVs I could see it being really difficult to ride safely around AVs. It’s fairly easy to anticipate actions of another human, and not as easy to anticipate when actions are decided by an algorithm. FWIW I think the above also applies to cyclists. (I suppose this becomes a non-issue if the assumption is that AVs will be so superior to human judgement as to never strike another motorcycle or cyclist - 5x safer sounds like a starting point)
  - ggreer 6y ago
    I own a Tesla and I ride a motorcycle. I would much prefer to deal with 100% autopiloted Teslas than current human drivers. Teslas don't have blind spots. They have eight cameras that give 360 degree views around the car. They also have a dozen ultrasonic sensors that can detect obstacles up to 5 meters in all directions. The only way to collide with a Tesla on autopilot is by doing something really dumb. In practice, a Tesla on autopilot tends to drive like a human taking a driving test: accelerating slowly, always signaling before turning or lane changing, always yielding to pedestrians, always braking or cancelling lane changes if an aggressive driver gets in the way, never honking. If traffic is too dense to lane change to the desired freeway exit, it reroutes rather than cutting into traffic (as pretty much any human would).
wffurr 6y ago
>> The challenge remains that people will be killed in accidents involving autonomous control. While that's almost certainly true (and depending on one's interpretation of autonomous is already true), some people already believe that zero deaths in transport is possible: https://visionzeronetwork.org/about/what-is-vision-zero/ https://visionzeronetwork.org/about/what-is-vision-zero/. If it's possible to hold human drivers to that standard, why not autonomous systems as well?
- bryanlarsen 6y ago
  That was my original hope for self-driving cars. The easiest way to ensure that you don't kill anybody is to limit your speed to 20mph. At and below that speed a car-pedestrian collision is highly unlikely to result in a dead pedestrian. Also, at 20mph you can stop on a dime. So I imagined a large fleet of robot cars traveling at 20mph and normalizing that speed, forcing human drivers to slow down too. But it turns out that self-driving vendors spend a lot of effort on "driving like a human", which includes driving faster than the speed limit and faster than vision zero would allow on shared streets & roads.
  - ghaff 6y ago
    Which could be done today. The vast majority of people obviously don't consider that acceptable in general.
- riversflow 6y ago
  Well that’s a silly vision. I have a vision of immortality too. This is a textbook example perfect being the enemy of good. I don’t want to hold any system to the standard of zero fatalities, and further I think this is why “shoot for the moon and even if you miss you’ll land amongst the stars” is faulty. People waste an exorbitant amount of time and fossil fuels commuting in passenger vehicles. Anything that significantly changes that balance should be considered. Progress is progress and letting everybody use the time they used to spend commuting is certainly progress.
  - bryanlarsen 6y ago
porknubbins 6y ago
Somehow the idea of a computer error killing me seems way worse than at least having a chance to save myself, since I’m a very cautious driver (though unlikely safer than average by 5x) . Self driving cars need to get to airline level safety where crashes are a rare thing and most people don’t think twice about giving up control to the pilots/auto pilot. If that takes expensive Lidar thats what we should use. I can’t imagine ever feeling good about trusting my life to a computer vision algorithm.
- sliken 6y ago
  Problem is, 95% [1] of drivers think they are better than average. [1] citation needed.
  - senko 6y ago
    Happy to oblige: https://www.smithlawco.com/blog/2017/december/do-most-drivers-really-think-they-are-above-aver/ https://www.smithlawco.com/blog/2017/december/do-most-driver... In summary: in a study done in 1980, 93% of Americans thought themselves better than an average driver.
    - tjoff 6y ago
      And they are all correct. Except for when they haven't slept, or aren't paying attention etc. That is not a good baseline to compare self-driving cars against. That would be horrific.
      - the8472 6y ago
        The average baseline involves a driver that is not paying attention or hasn't slept X% of the time. You cannot magically wave away all the bad days and pretend only the good humans are on the streets. The bad days happen, people die. The autonomous vehicle only has to do better than that. That's the whole point.
        tjoff 6y ago
njarboe 6y ago
I can't think of any product that has been developed since 1970 that can kill people. The exceptions are medical devices and pharmaceuticals. I sure hope self-driving cars can be an exception, but that will definitely take a federal law limiting the liability of manufactures. Similar to how small aircraft manufacturers were being pushed to extinction due to very high liability costs until the passage of the General Aviation Revitalization Act in 1994.
- ghaff 6y ago
  Lots of products can (and do) kill people. But drug side effects aside, it's hard to think of modern consumer products that, used and maintained properly, might just go and kill you some day and people being OK with that.
  - mcguire 6y ago
    To njarboe's point, cars, motorcycles, airplanes, and furniture predate 1970. As do most of the deadly chemical products.
- drjasonharrison 6y ago
  Boeing 737 Max? Ikea Malm dressers? Many products listed at https://www.cpsc.gov/Recalls https://www.cpsc.gov/Recalls
  - yadaeno 6y ago
    Add cars and motorcycles to this list.
    - njarboe 6y ago
      Cars, motorcycles, and airplanes existed before 1970.
- yadaeno 6y ago
  Cars?
foobarian 6y ago
> However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. This is a really good point. I drive really conservatively and like to think I will never ever cause an accident let alone a fatal one. I think if this lever was taken away I would have a hard time accepting the automated driving a significant amount of time.
throwaway2245 6y ago
> There will be cases where a court determines that the autonomous system was the cause. At the moment, manufacturers almost totally escape blame for fatal accidents [involving human drivers] - it's understood societally and in the legal system that the human driver was the one at fault. That isn't a totally accurate picture of the responsibility. The manufacturer provided a vehicle that included a risk of fatal accident. (Reducing this wrong and describing it as 'lives saved' feels uncomfortable to me) With an autonomous system, blame for fatalities can no longer be placed on a human driver: and yet, there is still a failure of responsibility (maybe this will be a more accurate placement of blame)
- CuriousPerson23 6y ago
  Hasn't there been only 1 fatal accident? If not, I can't imagine there have been >5, so seems unfair and misleading to make bold claims like that. I think the courts will rule, and if the system misunderstood something, the manufacturer will be at fault.
SoSoRoCoCo 6y ago
> I expect that in the end it will come down to a business decision, and that decision will be informed by an actuarial exercise: Will profits and insurance be able to cover the costs of defending and settling such cases. I'm seeing type of phrasing occur more and more. Once the defendant can be named in a legal action, we'll start seeing SDVs. IMHO, the worry isn't that they will kill, but that no one is to blame. Although it will change the day to day narrative of a pedestrian. E.g., My thought process will change from this person might not see me, to, that car's AI might not see me. ... or even "Oh, its a Toyota, they kill more than Hyundai... stand back!" But now I'm just writing SciFi.
- dvfjsdhgfv 6y ago
  > But now I'm just writing SciFi. I don't think so. Normally whenever I want to cross the street (as a pedestrian, but even more as a cyclist) and a car approaches I (unconsciously) examine its speed, and if it's higher than acceptable I try to make eye contact with the driver to make sure they see me and it's safe for me to go. How do I make contact with the AI of the car? More importantly, how do I get the cue I've been noticed?
  - SoSoRoCoCo 6y ago
    > I try to make eye contact That's a really good point. I forget how often when I'm walking, running, or biking I will try to make eye contact with a car to make sure we're aware of each other. Now how do I do that with an AI? More things we need to start thinking about!
  - tintor 6y ago
    That can be solved. Standardized "I see you and I am yielding" signal on front of all self-driving cars.
    - kaba0 6y ago
      There is a person in front of me, and the signal is on. Does it see me or only that other person?
HALtheWise 6y ago
It strikes me that a useful analogy here is the adoption of automatic elevators in buildings. In some ways, it's amazing that pretty much everyone in industrialized countries is OK with being locked in a windowless box controlled entirely by a computer, hanging over a hundreds of foot deep shaft, and in fact many people were terrified of that when Elevator operators were first replaced with computers. Some places even had operators employed to simply stand there and push the buttons to provide confidence that an trained expert was there, even though they didn't actually contribute to safety. Eventually, autonomous elevators got common enough that people will look at you really funny if you're not willing to ride in one, even though they are still responsible for ~20-30 deaths per year.
- dekervin 6y ago
  there is some kind of implied machine capability that a layman assign to a computer. If a task is culturally ( through movies , series, ... ) though to be within that implied capability people will be comfortable. ( cf automatic trains, ... )
- ggreer 6y ago
  Can you provide more info about people being terrified of automatic elevators? I searched around and everything I found seems to cite one NPR article from 2015.[1] The interviewee's book is out of print and costs $100[2], so that's where that trail stops. If public sentiment against automatic elevators was as strong as described, it seems like there would be more historical evidence available. It's easier for me to find articles disparaging self-checkout systems than for automated elevators. I realize the change in elevators happened long ago, making articles harder to find, but you'd think at least one of them would have gotten digitized and indexed. 1. https://www.npr.org/2015/07/31/427990392/remembering-when-driverless-elevators-drew-skepticism https://www.npr.org/2015/07/31/427990392/remembering-when-dr... 2. https://www.amazon.com/Ascending-Rooms-Express-Elevators-Passenger/dp/1937689107 https://www.amazon.com/Ascending-Rooms-Express-Elevators-Pas...
  - HALtheWise 6y ago
    I just spent some time digging, but finding original sources from the ~1950s is really hard without a NYTimes subscription. These sorts of things look like promising primary sources, but I can't access the full text. https://www.nytimes.com/1949/01/12/archives/city-gets-elevator-bill-isaacs-would-require-operator-in-buildings.html https://www.nytimes.com/1949/01/12/archives/city-gets-elevat... https://www.nytimes.com/1949/02/03/archives/tenants-want-a-doorman-suit-against-landlord-demands-elevators-with.html https://www.nytimes.com/1949/02/03/archives/tenants-want-a-d... https://www.nytimes.com/1928/11/11/archives/elevator-law-change-oppose-employment-of-operators-on-automatic.html https://www.nytimes.com/1928/11/11/archives/elevator-law-cha...
xenocyon 6y ago
> However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. So far it seems that this is very much the case. Autonomous cars do relatively well in highway scenarios whereas they appear to do poorly recognizing bicycles, for instance. Reducing safety to one single metric would be a big mistake.
BurningFrog 6y ago
Obviously the car manufacturer will be liable when a fault in their product kills people. And that is as it should be. The costs will be baked into the price of the vehicle. This aligns incentives. The manufacturer wants to pay as little as possible, and the owner will want to be safe. With the car recording video and other data, determining what actually happened should be simple and reliable. Note that the car owner no longer pays any insurance in this system.
- foobar1962 6y ago
  Smart manufacturers will cover this risk with third-party insurers, for some period of time: probably the vehicle warranty period. After that, it will be on the owner to renew. If I was one of these insurers, I'd be insisting the vehicles were maintained to an extremely high level of roadworthiness by mechanics that licensed or certified. Home DIY would be forbidden, as would any modification to the hardware or software. Think something like happens now for aircraft.
  - ghaff 6y ago
    It's reasonable to expect that the first generation or so of fully autonomous vehicles will be rented out and must adhere to a manufacturer-mandated update and service program or they can't be driven. You won't own them.
    - foobar1962 6y ago
      Ah, that's an interesting option. The car remains the property of the manufacturer who can control modifications and maintenance.
  - throwaway0a5e 6y ago
    You're gonna get instantly undercut by the insurers that can accurately price the risk rather than insist their customers jump through hoops.
  - BurningFrog 6y ago
    > After that, it will be on the owner to renew No one will pay to take responsibility for someone else's software bugs. That deal won't sell a single car.
paulryanrogers 6y ago
My guess is it's much easier for the public to understand the risks and liability of human drivers than AI. Humans often get distracted or misinterpret signals. And their errors stop with the driver. AI could react in more unexpected ways when it goes sideways. Humans too can get weird when chemically impaired but the liability is much easier to attribute.
jonplackett 6y ago
To add to the ‘different set of accidents’ point - the accidents caused by autonomous cars will most likely also be harder to accept. They will be accidents that a human driver (at least with the benefit of hindsight) would appear unlikely to have caused. Things like that Tesla driving straight into a wall and killing its driver. Or not seeing a van right in front of it because the sun was too bright. Things where a bit of common sense (not something AI excels at) might have avoided them. On the flip side the lives saved will be based on having super human reaction times / situational awareness and will be things no person would have been able to do. So maybe there’ll be a battle of public opinion (and PR!) weighing these things against each other.
- semi-extrinsic 6y ago
  No, if there are lives saved by self-driving cars, they will primarily be in instances where the driver was distracted/drunk/high. Thus any competent lawyer workin a case against a self-driving car company in the case where their car killed someone, will argue (rightly!) that those sane lives could also have been saved by less high-tech measures, such as AI detecting undesired human behaviour and putting the car in limp-home-mode or similar. In that case you are saving the same lives and not causing any new fatalities, but it's not a super cool tech that will attract lots of investment. So the choice to implement self driving was ultimately putting profits over human lives.
  - shkkmo 6y ago
    I think detecting impaired and distracted drivers is a lot harder than you think and more high tech since you need that detection in addition to the ability to limp home.
    - fiddlerwoaroof 6y ago
      There are very trivial situations where even current Teslas are safer than previous cars: while I wouldn’t trust my life to Autopilot, I’d rather drive next to a drunk driver using autopilot than one driving “manually”. Other simple things, like detecting excessive swerving and drifting out of the land, would be relatively straightforward signals to the car to maybe limit speed and encourage pulling over.
schoen 6y ago
> we anticipate that the number of people killed will be fewer, hence 'saving lives'. However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. There will be cases where a court determines that the autonomous system was the cause. Families of those killed will want justice, while those separately saved by autonomous systems may never be heard from in the same case. This reasoning was addressed with the VICP program for vaccines in the U.S., based on the idea that vaccines save many lives overall, but still injure a small number of people who will want to be compensated for that injury. https://www.hrsa.gov/vaccine-compensation/index.html https://www.hrsa.gov/vaccine-compensation/index.html Maybe there should be a self-driving-car injury compensation program along similar lines once particular cars are convincingly proven to be even moderately safer than human drivers. People might be mad about it, but maybe the precedent of the apparent success of the VICP would be persuasive (at least for courts and legislators, and maybe for some people actually injured by self-driving cars). A counterargument is that vaccine injuries are generally unforeseeable and reflect absolutely no fault on the part of the injured person (or injured person's guardian), while some people injured by self-driving cars will bear some fault for their own injuries, which insurers or manufacturers might well want the opportunity to adjudicate. (Otherwise, there could be a perverse incentive to deliberately do dangerous things around self-driving cars to provoke an injury and receive a payment, something I've heard already sometimes happens with personal injury claims against human drivers in some parts of the world.)
throwawaygh 6y ago
> Will profits and insurance be able to cover the costs of defending and settling such cases. Risk will be transferred to individual consumers through "pay attention / hands on wheels / you're still the driver" warnings. Those warnings will never go away, even with vehicles that are in all other ways marketed as L5.
- kaba0 6y ago
  And frankly, it should not be legal. Either don’t sell it as self-driving / don’t even have that feature, or take full responsibility (within normal limits) for any accident.
ehnto 6y ago
> However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. That's an interesting insight into part of the problem, thanks for pointing it out. Indeed in America if you ride a motorcycle but don't ride drunk, don't ride unlicensed, don't ride an unregistered bike, and do wear a helmet, you are orders of magnitude less likely to die because you are in a statistcally safer demographic of riders. Yet, any one of the riders in that safer demographic could still be one of the ones hit by a red light runner and killed without their blame. We would definitely feel that death more tragic and senseless.
sriku 6y ago
I'm wondering whether the first steps ought to be to make current driving much safer through automation - like detecting drunk driving, falling asleep, and such and limiting speed, refusing to start, alerting police and/or nearby vehicles etc. The lessons from that might better inform the long tail of cases that autonomous vehicles will eventually face?
- visarga 6y ago
  Yeah, people won't buy 'crippled' car brands. Probably no manufacturer will dare be first.
  - amanaplanacanal 6y ago
    I could picture legal requirements that cars check for say, alcohol consumption though. Then they wouldn’t have a choice.

tgv 6y ago

Idk about risk: that is hard to establish. There are so many conditions in which an automatic pilot hasn't been tested. We don't even know the factors involved in estimating the risk: e.g., is it dependent on the human co-pilot? And self-driving cars may change the car usage patterns, exerting a contextual influence on the risk. Then there's the question of responsibility. Who will be held responsible when the automatic pilot is driving? If it's the human, then a high risk of causing an accident will be unacceptable to many drivers.

Justsignedup 6y ago

Sensationalism will win out quite a bit. And responsibility. Its just a tough problem. - People developing Bell's Palsy at the same rate with vaccine than not. But suddenly only those with the vaccine show up in the social media feeds. Because nobody just posts "I just got bell's palsy" but now they do because people are paying attention. Same will happen with AVs. "I just got into an AV accident" will make headlines, while "I just dozed off and hit a kid" will barely circulate. - People inherently trust humans over technology. Just because. So they will be quick to distrust autonomous vehicles. I already had convos about the fact that yes, teslas do kill, but on the whole self-driving teslas do kill far less than non. - When a human drives, the liability is on the human. When a car self-drives, the liability might be on the manufacturer.

SkyBelow 6y ago
>When a human drives, the liability is on the human. When a car self-drives, the liability might be on the manufacturer. I think this is a point that needs more emphasis, especially on the word 'might'. Without both laws and a history of court cases giving evidence for how those laws are interpreted and enforced, it isn't possible to tell where liability may end up. I wonder if that is part of the reason people are hesitant. Liability is being removed from the driver, but it doesn't see have have found a place to settle back down at, so people are viewing it as if liability is just being removed. For initial court cases (and the amount of time and money it takes to fight them), this may not be an unrealistic expectation.
- ghaff 6y ago
  If liability is on anyone, it would seem it has to be the manufacturer. And if there's no liability then the options are basically set up some sort of vaccine fund-like system or just to shrug and say it's between you and the insurance company.
cj 6y ago
> People inherently trust humans over technology. Just because. From the perspective of someone who rides a motorcycle, the #1 thing you need to do to not crash is to anticipate what all vehicles around you might possibly do. For example, I always avoid riding in another car’s blind spot for obvious reasons. The problem (for motorcyclists) will now be trying to adapt to understand what a Tesla might do and where a Tesla’s blind spots might be - and once you add in the idiosyncrasies of other AVs I could see it being really difficult to ride safely around AVs. It’s fairly easy to anticipate actions of another human, and not as easy to anticipate when actions are decided by an algorithm. FWIW I think the above also applies to cyclists. (I suppose this becomes a non-issue if the assumption is that AVs will be so superior to human judgement as to never strike another motorcycle or cyclist - 5x safer sounds like a starting point)
- ggreer 6y ago
  I own a Tesla and I ride a motorcycle. I would much prefer to deal with 100% autopiloted Teslas than current human drivers. Teslas don't have blind spots. They have eight cameras that give 360 degree views around the car. They also have a dozen ultrasonic sensors that can detect obstacles up to 5 meters in all directions. The only way to collide with a Tesla on autopilot is by doing something really dumb. In practice, a Tesla on autopilot tends to drive like a human taking a driving test: accelerating slowly, always signaling before turning or lane changing, always yielding to pedestrians, always braking or cancelling lane changes if an aggressive driver gets in the way, never honking. If traffic is too dense to lane change to the desired freeway exit, it reroutes rather than cutting into traffic (as pretty much any human would).

wffurr 6y ago

>> The challenge remains that people will be killed in accidents involving autonomous control. While that's almost certainly true (and depending on one's interpretation of autonomous is already true), some people already believe that zero deaths in transport is possible: https://visionzeronetwork.org/about/what-is-vision-zero/ https://visionzeronetwork.org/about/what-is-vision-zero/. If it's possible to hold human drivers to that standard, why not autonomous systems as well?

bryanlarsen 6y ago
That was my original hope for self-driving cars. The easiest way to ensure that you don't kill anybody is to limit your speed to 20mph. At and below that speed a car-pedestrian collision is highly unlikely to result in a dead pedestrian. Also, at 20mph you can stop on a dime. So I imagined a large fleet of robot cars traveling at 20mph and normalizing that speed, forcing human drivers to slow down too. But it turns out that self-driving vendors spend a lot of effort on "driving like a human", which includes driving faster than the speed limit and faster than vision zero would allow on shared streets & roads.
- ghaff 6y ago
  Which could be done today. The vast majority of people obviously don't consider that acceptable in general.
riversflow 6y ago
Well that’s a silly vision. I have a vision of immortality too. This is a textbook example perfect being the enemy of good. I don’t want to hold any system to the standard of zero fatalities, and further I think this is why “shoot for the moon and even if you miss you’ll land amongst the stars” is faulty. People waste an exorbitant amount of time and fossil fuels commuting in passenger vehicles. Anything that significantly changes that balance should be considered. Progress is progress and letting everybody use the time they used to spend commuting is certainly progress.
- bryanlarsen 6y ago

porknubbins 6y ago

Somehow the idea of a computer error killing me seems way worse than at least having a chance to save myself, since I’m a very cautious driver (though unlikely safer than average by 5x) . Self driving cars need to get to airline level safety where crashes are a rare thing and most people don’t think twice about giving up control to the pilots/auto pilot. If that takes expensive Lidar thats what we should use. I can’t imagine ever feeling good about trusting my life to a computer vision algorithm.

sliken 6y ago
Problem is, 95% [1] of drivers think they are better than average. [1] citation needed.
- senko 6y ago
  Happy to oblige: https://www.smithlawco.com/blog/2017/december/do-most-drivers-really-think-they-are-above-aver/ https://www.smithlawco.com/blog/2017/december/do-most-driver... In summary: in a study done in 1980, 93% of Americans thought themselves better than an average driver.
  - tjoff 6y ago
    And they are all correct. Except for when they haven't slept, or aren't paying attention etc. That is not a good baseline to compare self-driving cars against. That would be horrific.
    - the8472 6y ago
      The average baseline involves a driver that is not paying attention or hasn't slept X% of the time. You cannot magically wave away all the bad days and pretend only the good humans are on the streets. The bad days happen, people die. The autonomous vehicle only has to do better than that. That's the whole point.
      - tjoff 6y ago

njarboe 6y ago

I can't think of any product that has been developed since 1970 that can kill people. The exceptions are medical devices and pharmaceuticals. I sure hope self-driving cars can be an exception, but that will definitely take a federal law limiting the liability of manufactures. Similar to how small aircraft manufacturers were being pushed to extinction due to very high liability costs until the passage of the General Aviation Revitalization Act in 1994.

ghaff 6y ago
Lots of products can (and do) kill people. But drug side effects aside, it's hard to think of modern consumer products that, used and maintained properly, might just go and kill you some day and people being OK with that.
- mcguire 6y ago
  To njarboe's point, cars, motorcycles, airplanes, and furniture predate 1970. As do most of the deadly chemical products.
drjasonharrison 6y ago
Boeing 737 Max? Ikea Malm dressers? Many products listed at https://www.cpsc.gov/Recalls https://www.cpsc.gov/Recalls
- yadaeno 6y ago
  Add cars and motorcycles to this list.
  - njarboe 6y ago
    Cars, motorcycles, and airplanes existed before 1970.
yadaeno 6y ago
Cars?

foobarian 6y ago

> However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. This is a really good point. I drive really conservatively and like to think I will never ever cause an accident let alone a fatal one. I think if this lever was taken away I would have a hard time accepting the automated driving a significant amount of time.

throwaway2245 6y ago

> There will be cases where a court determines that the autonomous system was the cause. At the moment, manufacturers almost totally escape blame for fatal accidents [involving human drivers] - it's understood societally and in the legal system that the human driver was the one at fault. That isn't a totally accurate picture of the responsibility. The manufacturer provided a vehicle that included a risk of fatal accident. (Reducing this wrong and describing it as 'lives saved' feels uncomfortable to me) With an autonomous system, blame for fatalities can no longer be placed on a human driver: and yet, there is still a failure of responsibility (maybe this will be a more accurate placement of blame)

CuriousPerson23 6y ago
Hasn't there been only 1 fatal accident? If not, I can't imagine there have been >5, so seems unfair and misleading to make bold claims like that. I think the courts will rule, and if the system misunderstood something, the manufacturer will be at fault.

SoSoRoCoCo 6y ago

> I expect that in the end it will come down to a business decision, and that decision will be informed by an actuarial exercise: Will profits and insurance be able to cover the costs of defending and settling such cases. I'm seeing type of phrasing occur more and more. Once the defendant can be named in a legal action, we'll start seeing SDVs. IMHO, the worry isn't that they will kill, but that no one is to blame. Although it will change the day to day narrative of a pedestrian. E.g., My thought process will change from this person might not see me, to, that car's AI might not see me. ... or even "Oh, its a Toyota, they kill more than Hyundai... stand back!" But now I'm just writing SciFi.

dvfjsdhgfv 6y ago
> But now I'm just writing SciFi. I don't think so. Normally whenever I want to cross the street (as a pedestrian, but even more as a cyclist) and a car approaches I (unconsciously) examine its speed, and if it's higher than acceptable I try to make eye contact with the driver to make sure they see me and it's safe for me to go. How do I make contact with the AI of the car? More importantly, how do I get the cue I've been noticed?
- SoSoRoCoCo 6y ago
  > I try to make eye contact That's a really good point. I forget how often when I'm walking, running, or biking I will try to make eye contact with a car to make sure we're aware of each other. Now how do I do that with an AI? More things we need to start thinking about!
- tintor 6y ago
  That can be solved. Standardized "I see you and I am yielding" signal on front of all self-driving cars.
  - kaba0 6y ago
    There is a person in front of me, and the signal is on. Does it see me or only that other person?

HALtheWise 6y ago

It strikes me that a useful analogy here is the adoption of automatic elevators in buildings. In some ways, it's amazing that pretty much everyone in industrialized countries is OK with being locked in a windowless box controlled entirely by a computer, hanging over a hundreds of foot deep shaft, and in fact many people were terrified of that when Elevator operators were first replaced with computers. Some places even had operators employed to simply stand there and push the buttons to provide confidence that an trained expert was there, even though they didn't actually contribute to safety. Eventually, autonomous elevators got common enough that people will look at you really funny if you're not willing to ride in one, even though they are still responsible for ~20-30 deaths per year.

dekervin 6y ago
there is some kind of implied machine capability that a layman assign to a computer. If a task is culturally ( through movies , series, ... ) though to be within that implied capability people will be comfortable. ( cf automatic trains, ... )
ggreer 6y ago
Can you provide more info about people being terrified of automatic elevators? I searched around and everything I found seems to cite one NPR article from 2015.[1] The interviewee's book is out of print and costs $100[2], so that's where that trail stops. If public sentiment against automatic elevators was as strong as described, it seems like there would be more historical evidence available. It's easier for me to find articles disparaging self-checkout systems than for automated elevators. I realize the change in elevators happened long ago, making articles harder to find, but you'd think at least one of them would have gotten digitized and indexed. 1. https://www.npr.org/2015/07/31/427990392/remembering-when-driverless-elevators-drew-skepticism https://www.npr.org/2015/07/31/427990392/remembering-when-dr... 2. https://www.amazon.com/Ascending-Rooms-Express-Elevators-Passenger/dp/1937689107 https://www.amazon.com/Ascending-Rooms-Express-Elevators-Pas...
- HALtheWise 6y ago
  I just spent some time digging, but finding original sources from the ~1950s is really hard without a NYTimes subscription. These sorts of things look like promising primary sources, but I can't access the full text. https://www.nytimes.com/1949/01/12/archives/city-gets-elevator-bill-isaacs-would-require-operator-in-buildings.html https://www.nytimes.com/1949/01/12/archives/city-gets-elevat... https://www.nytimes.com/1949/02/03/archives/tenants-want-a-doorman-suit-against-landlord-demands-elevators-with.html https://www.nytimes.com/1949/02/03/archives/tenants-want-a-d... https://www.nytimes.com/1928/11/11/archives/elevator-law-change-oppose-employment-of-operators-on-automatic.html https://www.nytimes.com/1928/11/11/archives/elevator-law-cha...

xenocyon 6y ago

> However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. So far it seems that this is very much the case. Autonomous cars do relatively well in highway scenarios whereas they appear to do poorly recognizing bicycles, for instance. Reducing safety to one single metric would be a big mistake.

BurningFrog 6y ago

Obviously the car manufacturer will be liable when a fault in their product kills people. And that is as it should be. The costs will be baked into the price of the vehicle. This aligns incentives. The manufacturer wants to pay as little as possible, and the owner will want to be safe. With the car recording video and other data, determining what actually happened should be simple and reliable. Note that the car owner no longer pays any insurance in this system.

foobar1962 6y ago
Smart manufacturers will cover this risk with third-party insurers, for some period of time: probably the vehicle warranty period. After that, it will be on the owner to renew. If I was one of these insurers, I'd be insisting the vehicles were maintained to an extremely high level of roadworthiness by mechanics that licensed or certified. Home DIY would be forbidden, as would any modification to the hardware or software. Think something like happens now for aircraft.
- ghaff 6y ago
  It's reasonable to expect that the first generation or so of fully autonomous vehicles will be rented out and must adhere to a manufacturer-mandated update and service program or they can't be driven. You won't own them.
  - foobar1962 6y ago
    Ah, that's an interesting option. The car remains the property of the manufacturer who can control modifications and maintenance.
- throwaway0a5e 6y ago
  You're gonna get instantly undercut by the insurers that can accurately price the risk rather than insist their customers jump through hoops.
- BurningFrog 6y ago
  > After that, it will be on the owner to renew No one will pay to take responsibility for someone else's software bugs. That deal won't sell a single car.

paulryanrogers 6y ago

My guess is it's much easier for the public to understand the risks and liability of human drivers than AI. Humans often get distracted or misinterpret signals. And their errors stop with the driver. AI could react in more unexpected ways when it goes sideways. Humans too can get weird when chemically impaired but the liability is much easier to attribute.

jonplackett 6y ago

To add to the ‘different set of accidents’ point - the accidents caused by autonomous cars will most likely also be harder to accept. They will be accidents that a human driver (at least with the benefit of hindsight) would appear unlikely to have caused. Things like that Tesla driving straight into a wall and killing its driver. Or not seeing a van right in front of it because the sun was too bright. Things where a bit of common sense (not something AI excels at) might have avoided them. On the flip side the lives saved will be based on having super human reaction times / situational awareness and will be things no person would have been able to do. So maybe there’ll be a battle of public opinion (and PR!) weighing these things against each other.

semi-extrinsic 6y ago
No, if there are lives saved by self-driving cars, they will primarily be in instances where the driver was distracted/drunk/high. Thus any competent lawyer workin a case against a self-driving car company in the case where their car killed someone, will argue (rightly!) that those sane lives could also have been saved by less high-tech measures, such as AI detecting undesired human behaviour and putting the car in limp-home-mode or similar. In that case you are saving the same lives and not causing any new fatalities, but it's not a super cool tech that will attract lots of investment. So the choice to implement self driving was ultimately putting profits over human lives.
- shkkmo 6y ago
  I think detecting impaired and distracted drivers is a lot harder than you think and more high tech since you need that detection in addition to the ability to limp home.
  - fiddlerwoaroof 6y ago
    There are very trivial situations where even current Teslas are safer than previous cars: while I wouldn’t trust my life to Autopilot, I’d rather drive next to a drunk driver using autopilot than one driving “manually”. Other simple things, like detecting excessive swerving and drifting out of the land, would be relatively straightforward signals to the car to maybe limit speed and encourage pulling over.

schoen 6y ago

> we anticipate that the number of people killed will be fewer, hence 'saving lives'. However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. There will be cases where a court determines that the autonomous system was the cause. Families of those killed will want justice, while those separately saved by autonomous systems may never be heard from in the same case. This reasoning was addressed with the VICP program for vaccines in the U.S., based on the idea that vaccines save many lives overall, but still injure a small number of people who will want to be compensated for that injury. https://www.hrsa.gov/vaccine-compensation/index.html https://www.hrsa.gov/vaccine-compensation/index.html Maybe there should be a self-driving-car injury compensation program along similar lines once particular cars are convincingly proven to be even moderately safer than human drivers. People might be mad about it, but maybe the precedent of the apparent success of the VICP would be persuasive (at least for courts and legislators, and maybe for some people actually injured by self-driving cars). A counterargument is that vaccine injuries are generally unforeseeable and reflect absolutely no fault on the part of the injured person (or injured person's guardian), while some people injured by self-driving cars will bear some fault for their own injuries, which insurers or manufacturers might well want the opportunity to adjudicate. (Otherwise, there could be a perverse incentive to deliberately do dangerous things around self-driving cars to provoke an injury and receive a payment, something I've heard already sometimes happens with personal injury claims against human drivers in some parts of the world.)

throwawaygh 6y ago

> Will profits and insurance be able to cover the costs of defending and settling such cases. Risk will be transferred to individual consumers through "pay attention / hands on wheels / you're still the driver" warnings. Those warnings will never go away, even with vehicles that are in all other ways marketed as L5.

kaba0 6y ago
And frankly, it should not be legal. Either don’t sell it as self-driving / don’t even have that feature, or take full responsibility (within normal limits) for any accident.

ehnto 6y ago

> However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. That's an interesting insight into part of the problem, thanks for pointing it out. Indeed in America if you ride a motorcycle but don't ride drunk, don't ride unlicensed, don't ride an unregistered bike, and do wear a helmet, you are orders of magnitude less likely to die because you are in a statistcally safer demographic of riders. Yet, any one of the riders in that safer demographic could still be one of the ones hit by a red light runner and killed without their blame. We would definitely feel that death more tragic and senseless.

sriku 6y ago

I'm wondering whether the first steps ought to be to make current driving much safer through automation - like detecting drunk driving, falling asleep, and such and limiting speed, refusing to start, alerting police and/or nearby vehicles etc. The lessons from that might better inform the long tail of cases that autonomous vehicles will eventually face?

visarga 6y ago
Yeah, people won't buy 'crippled' car brands. Probably no manufacturer will dare be first.
- amanaplanacanal 6y ago
  I could picture legal requirements that cars check for say, alcohol consumption though. Then they wouldn’t have a choice.