10 ms·

This is potentially precedent as a legal blocker for public adoption of self-driving cars. If I am to serve a 1 year prison sentence for faulty code or a deprec

by vichu 8y ago

This is potentially precedent as a legal blocker for public adoption of self-driving cars. If I am to serve a 1 year prison sentence for faulty code or a deprecated LIDAR sensor, I don't see in what scenario I would be willing to leave my fate in the hands of a self-driving car manufacturer. Just as in the case of a chauffeur, if they are driving and commit vehicular manslaughter, I would not expect to be deemed guilty. It seems to me that it would be the benefit of self-driving car companies to own up to liability as it is to their benefit of achieving widespread adoption. For example, Volvo is in line with this idea and has publicly stated that they would accept full liability in fully autonomous operation modes[0]. In this case, I do think that some liability lies with the driver - as they were tasked specifically with prevent situations like this. What is not clear is whether or not this task is even humanly feasible given reaction times, and based off of this, whether or not Uber has been criminally negligible. Given this, I am surprised that the prosecutor seems to have absolved Uber of any blame. [0] https://www.media.volvocars.com/global/en-gb/media/pressreleases/167975/us-urged-to-establish-nationwide-federal-guidelines-for-autonomous-driving https://www.media.volvocars.com/global/en-gb/media/pressrele...

Symmetry 8y ago
I'd really like to see self-driving car liability treated the same way airline crashes are. Less "whose fault is this" and more "how can we prevent this same accident from ever happening again?" The second is the really important question and the one we need to keep answering to reduce the toll of driving deaths. This will mean the cars driving on our roads getting more and more regulated over time but I'm quite OK with that.
- freddie_mercury 8y ago
  Airline crashes absolutely are treated as "whose fault is this". That's why airline crashes usually have lots of lawsuits, with millions of dollars of damages, filed after they happen. Every NTSB report has a "probable cause" section determining who or what was at fault. And then the FAA hands out fines and courts award additional damages largely based on that finding. It is true that the punishments aren't handled by the NTSB but that's more about separation of concerns & departmental remits than anything else. Is separating out "fault finding" from "punishments" a good idea? Maybe? I honestly don't know.
  - lawguy 8y ago
    > Every NTSB report has a "probable cause" section determining who or what was at fault. And then the FAA hands out fines and courts award additional damages largely based on that finding. In case everyone is not aware, NTSB reports cannot be used in court to establish fault or otherwise be used as evidence when seeking compensation for damages. "No part of a report of the Board, related to an accident or an investigation of an accident, may be admitted into evidence or used in a civil action for damages resulting from a matter mentioned in the report." 49 USC 1154(b)
    - EGreg 8y ago
      I have a question.. In criminal cases what happens with inadmissible evidence? Like fruit of poisonous tree doctrine... what if you literally have video proof and tons of other graphic proof of a murder, you really just ignore it and let the perp go back into society?
      - Rylinks 8y ago
        if you have tons of proof about a murder, it's almost certain to fall under one of the exceptions to the doctrine. e.g, if law enforcement would have discovered the evidence anyway, it is not excluded.
      - ghaff 8y ago
        In principle, it’s inadmissible. So, theoretically, yes. The perp just goes free. Not very common for a lot of reasons.
      - admax88q 8y ago
        There are exceptions to the doctrine for exactly the reasons you mention. The legal system isn't as insane as people like to make it out to be.
  - tehlike 8y ago
    The part you missed is probably the post mortem investigations are why air travel is the safest form of transportation.
- bellerose 8y ago
  > "how can we prevent this same accident from ever happening again?" It would be nice if society adopted this with all crimes. It's not like we can prove we're any different than the low level design of a machine; when it comes to the "free will" ideology currently the justice system is designed around.
  - moises_silva 8y ago
    This. For me believing in free will is just another type of religion. It makes us feel safer and that we're in control. I think is useful for that, but we're deluding ourselves. And I just realized you're being downvoted. How surprising.
    - michaelmrose 8y ago
      Even if free will is an illusion it my be maladaptive to design a justice system without consequences for perceived faults.
      - bellerose 8y ago
        It may be the downfall of humanity progress as well and things may change.
      - moises_silva 8y ago
        I am not advocating a 'justice' system without consequences. You will have consequences. Consequences are just another input into our brains that do help deter some people from some behaviours. I don't kill people because I'd go to jail. I don't kill people because I don't want to hurt them (and that is on my programming, upbringing, possibly nature to a degree as well). I think the only way I'd kill someone is by accident or in a spur of the moment when emotionally disturbed by an incident (me or my family fear for our lives). Society has mostly accounted for those cases and I may not even go to jail if I did it, because people largely can relate to those situations and they agree it's not 'fair' send me to jail for them. However, there's several sectors of the population no one seems to be able or want to relate to (poor people that got little opportunities and guidance in life, abused people, mentally ill, psychopaths, a mix of all of that etc), and the people creating the rules therefore decide to 'punish' them in the name of justice for being bad and because we have not devised (and not invested enough time either) a better and more humane way to deal with them. In theory we want them to be reformed, in practice, is mostly about retribution and putting undesirables away.
ndespres 8y ago
Operators of vehicles need to be held personally responsible for the damages caused by putting their vehicles on the road, whether the vehicle is autonomous or not. Additionally there must be a responsibility for manufacturers to not create faulty products, but that can not absolve the "operator" of the product. It should not be said that the "operator" of a Volvo autonomous vehicle is not responsible for the actions of the vehicle under her control, simply because Volvo says they accept full liability. I think your chauffeur analogy is flawed. If I set cruise control in my car to 75mph in a 25mph school zone, am I not liable? It is not possible to delegate our personal responsibilities to machines, and wash our hands of it. Volvo's acceptance of liability seems little more than a smart business decision- the cost of an insurance payout to the family of the deceased is less than the money they stand to earn in profit from selling autonomous vehicles. We can't allow this generous actuarial calculation to absolve the operator of such a vehicle of responsibility too. If you don't like it, take the train.
- smallgovt 8y ago
  Why?
  - ndespres 8y ago
    The individual operator (the driver, or the person in the operator seat who presses the "engage autonomous mode" on the car) is the one who chose to buy a vehicle and put it on the road, potentially endangering other people. We need to be responsible for our own actions and the situations we create.
    - thereisnospork 8y ago
      Is their a gun shop owner liable for murder (and/or wrongful death) every time someone is killed with a firearm? What about the clerks at the gun store? Or the truck driver who delivered the firearm to the store? All of their respective actions clearly created a situation where people have died; your position is silly.
      - 8y ago
Klathmon 8y ago
>This is potentially precedent as a legal blocker for public adoption of self-driving cars. I don't think it is because in this case the person was hired by Uber specifically to "drive" this car. They knew that the car is still an incomplete product, they knew that they were testing it, they knew that they were literally getting paid to keep this car from doing something like this. That's very different than a random person using a self-driving car that they expect and were told is a finished product.
- masklinn 8y ago
  > I don't think it is because in this case the person was hired by Uber specifically to "drive" this car. They knew that the car is still an incomplete product, they knew that they were testing it, they knew that they were literally getting paid to keep this car from doing something like this. Sadly, while that is in fact the task of the safety drivers in other automated car trials it wasn't the case here for uber, because the "safety driver" was given both the job of safety driving and of checking, controlling and classifying the automated system's information, ensuring they'd have to split their focus and would need to frequently context-switch between two completely different tasks. Worse, this decision of Uber all but ensured this would happen: the automation system is more likely to lose its shit and need checking specifically in situations where the safety driver would be most useful.
  - Klathmon 8y ago
    Don't get me wrong, Uber isn't fault free here in my opinion, and I'm mostly in aggreement with you. Pretty much everyone involved here was negligent in some way if you ask me. My point was only that I don't think this would set any kind of precedent since it's so far from a simple case of "an owner of a self driving car".
mr_toad 8y ago
> What is not clear is whether or not this task is even humanly feasible Not watching TV while you’re supposed to be driving is a pretty low bar. But apparently, humans can’t even be relied on to do this, even when it’s their one single job.
- acdha 8y ago
  If you think it’s easy, go sit in the passenger seat without talking or using your phone and see how long you can maintain focus. There are decades of experiments showing that boredom and fatigue are huge problems for this kind of work where someone is mostly idle except for rare events. It’s why the TSA mixes fake images into the stream on X-ray scans so people see things regularly and don’t zone out. Anyone smart asking humans to do repetitive work builds layers of safeguards in to keep people engaged – or in Uber’s case they just hope they can shift blame to the person in a bad design, which worked so far.
  - cma 8y ago
    Uber should have had people remote-monitoring the monitors at this early stage.
    - acdha 8y ago
      I saw a different company’s car today: they’re being responsible and have two people in the car.
      - masklinn 8y ago
        Pretty much all of them do. In fact Uber used to do that before they moved to AZ. Having a single person responsible for driving & monitoring was a way to save cost and log more miles.
  - cf498 8y ago
    Also >"The system is not designed to alert the operator," the report notes.
usaphp 8y ago
> If I am to serve a 1 year prison sentence for faulty code I wonder why would a programmer go and work on a code when he is always in danger of going to prison for every mistake he makes. We all know that bugs in software happen and it’s impossible to write bug free code
- Alex3917 8y ago
  > I wonder why would a programmer go and work on a code when he is always in danger of going to prison for every mistake he makes. You don't go to prison for "mistakes", you go to prison for criminal negligence. Given that you can go to prison for criminal negligence in any other profession (e.g. a dentist who exposes thousands of people to HIV), why would programmers be the one category of job that's exempt from criminal laws?
  - jdminhbg 8y ago
    > You don't go to prison for "mistakes", you go to prison for criminal negligence. The difference between the two is what a politically-motivated prosecutor can convince a jury of.
    - vkou 8y ago
      If a politically motivated prosecutor can convince a jury of twelve people that Bill killed Dave, by using his psychic brain powers, or, alternatively, by sticking pins into a voodoo doll, or some other form of non-scientific witchcraft, Bill is going to hang. The hypothetical gullibility of juries doesn't mean that we shouldn't have laws against murder, juries, or prosecutors.
      - jdminhbg 8y ago
        The line between mistake and negligence is a lot blurrier than the line between killed and didn’t kill.
- SiempreViernes 8y ago
  If the programmer can’tbe bothered to put in enough verification effort to avoid being negligent, they should not be in charge of safety critical code. Plainly: don’t pretend there aren’t ways to ensure correctness of systems.
Dylan16807 8y ago
The code wasn't faulty. The configuration was explicitly set to not brake, as far as I understand. If someone got in trouble here, it wouldn't be for programming.
wmeredith 8y ago
I know it’s glib, but just on the face of it, I find the idea of Uber taking responsibility for anything a laughable proposition.
roenxi 8y ago
> If I am to serve a 1 year prison sentence for faulty code or a deprecated LIDAR sensor, I don't see in what scenario I would be willing to leave my fate in the hands of a self-driving car manufacturer. What if we came up with a reasonably objective test of driver skill and the car outperforms you? This is and remains the standard argument - the car doesn't have to be perfect, it just has to beat a human having a bad day. If you are worried about the liability, maybe keep the car serviced and sit in the drivers seat and look out of the window instead of watching a movie. Tell the car to dive slowly maybe. I feel sorry for the driver who is creating this precedent, but this is hardly a road block. The average human is not a great driver.
- deleted 8y ago
  [deleted]

by vichu 8y ago

Symmetry 8y ago

I'd really like to see self-driving car liability treated the same way airline crashes are. Less "whose fault is this" and more "how can we prevent this same accident from ever happening again?" The second is the really important question and the one we need to keep answering to reduce the toll of driving deaths. This will mean the cars driving on our roads getting more and more regulated over time but I'm quite OK with that.

freddie_mercury 8y ago
Airline crashes absolutely are treated as "whose fault is this". That's why airline crashes usually have lots of lawsuits, with millions of dollars of damages, filed after they happen. Every NTSB report has a "probable cause" section determining who or what was at fault. And then the FAA hands out fines and courts award additional damages largely based on that finding. It is true that the punishments aren't handled by the NTSB but that's more about separation of concerns & departmental remits than anything else. Is separating out "fault finding" from "punishments" a good idea? Maybe? I honestly don't know.
- lawguy 8y ago
  > Every NTSB report has a "probable cause" section determining who or what was at fault. And then the FAA hands out fines and courts award additional damages largely based on that finding. In case everyone is not aware, NTSB reports cannot be used in court to establish fault or otherwise be used as evidence when seeking compensation for damages. "No part of a report of the Board, related to an accident or an investigation of an accident, may be admitted into evidence or used in a civil action for damages resulting from a matter mentioned in the report." 49 USC 1154(b)
  - EGreg 8y ago
    I have a question.. In criminal cases what happens with inadmissible evidence? Like fruit of poisonous tree doctrine... what if you literally have video proof and tons of other graphic proof of a murder, you really just ignore it and let the perp go back into society?
    - Rylinks 8y ago
      if you have tons of proof about a murder, it's almost certain to fall under one of the exceptions to the doctrine. e.g, if law enforcement would have discovered the evidence anyway, it is not excluded.
    - ghaff 8y ago
      In principle, it’s inadmissible. So, theoretically, yes. The perp just goes free. Not very common for a lot of reasons.
    - admax88q 8y ago
      There are exceptions to the doctrine for exactly the reasons you mention. The legal system isn't as insane as people like to make it out to be.
- tehlike 8y ago
  The part you missed is probably the post mortem investigations are why air travel is the safest form of transportation.
bellerose 8y ago
> "how can we prevent this same accident from ever happening again?" It would be nice if society adopted this with all crimes. It's not like we can prove we're any different than the low level design of a machine; when it comes to the "free will" ideology currently the justice system is designed around.
- moises_silva 8y ago
  This. For me believing in free will is just another type of religion. It makes us feel safer and that we're in control. I think is useful for that, but we're deluding ourselves. And I just realized you're being downvoted. How surprising.
  - michaelmrose 8y ago
    Even if free will is an illusion it my be maladaptive to design a justice system without consequences for perceived faults.
    - bellerose 8y ago
      It may be the downfall of humanity progress as well and things may change.
    - moises_silva 8y ago
      I am not advocating a 'justice' system without consequences. You will have consequences. Consequences are just another input into our brains that do help deter some people from some behaviours. I don't kill people because I'd go to jail. I don't kill people because I don't want to hurt them (and that is on my programming, upbringing, possibly nature to a degree as well). I think the only way I'd kill someone is by accident or in a spur of the moment when emotionally disturbed by an incident (me or my family fear for our lives). Society has mostly accounted for those cases and I may not even go to jail if I did it, because people largely can relate to those situations and they agree it's not 'fair' send me to jail for them. However, there's several sectors of the population no one seems to be able or want to relate to (poor people that got little opportunities and guidance in life, abused people, mentally ill, psychopaths, a mix of all of that etc), and the people creating the rules therefore decide to 'punish' them in the name of justice for being bad and because we have not devised (and not invested enough time either) a better and more humane way to deal with them. In theory we want them to be reformed, in practice, is mostly about retribution and putting undesirables away.

ndespres 8y ago

Operators of vehicles need to be held personally responsible for the damages caused by putting their vehicles on the road, whether the vehicle is autonomous or not. Additionally there must be a responsibility for manufacturers to not create faulty products, but that can not absolve the "operator" of the product. It should not be said that the "operator" of a Volvo autonomous vehicle is not responsible for the actions of the vehicle under her control, simply because Volvo says they accept full liability. I think your chauffeur analogy is flawed. If I set cruise control in my car to 75mph in a 25mph school zone, am I not liable? It is not possible to delegate our personal responsibilities to machines, and wash our hands of it. Volvo's acceptance of liability seems little more than a smart business decision- the cost of an insurance payout to the family of the deceased is less than the money they stand to earn in profit from selling autonomous vehicles. We can't allow this generous actuarial calculation to absolve the operator of such a vehicle of responsibility too. If you don't like it, take the train.

smallgovt 8y ago
Why?
- ndespres 8y ago
  The individual operator (the driver, or the person in the operator seat who presses the "engage autonomous mode" on the car) is the one who chose to buy a vehicle and put it on the road, potentially endangering other people. We need to be responsible for our own actions and the situations we create.
  - thereisnospork 8y ago
    Is their a gun shop owner liable for murder (and/or wrongful death) every time someone is killed with a firearm? What about the clerks at the gun store? Or the truck driver who delivered the firearm to the store? All of their respective actions clearly created a situation where people have died; your position is silly.
    - 8y ago

Klathmon 8y ago

>This is potentially precedent as a legal blocker for public adoption of self-driving cars. I don't think it is because in this case the person was hired by Uber specifically to "drive" this car. They knew that the car is still an incomplete product, they knew that they were testing it, they knew that they were literally getting paid to keep this car from doing something like this. That's very different than a random person using a self-driving car that they expect and were told is a finished product.

masklinn 8y ago
> I don't think it is because in this case the person was hired by Uber specifically to "drive" this car. They knew that the car is still an incomplete product, they knew that they were testing it, they knew that they were literally getting paid to keep this car from doing something like this. Sadly, while that is in fact the task of the safety drivers in other automated car trials it wasn't the case here for uber, because the "safety driver" was given both the job of safety driving and of checking, controlling and classifying the automated system's information, ensuring they'd have to split their focus and would need to frequently context-switch between two completely different tasks. Worse, this decision of Uber all but ensured this would happen: the automation system is more likely to lose its shit and need checking specifically in situations where the safety driver would be most useful.
- Klathmon 8y ago
  Don't get me wrong, Uber isn't fault free here in my opinion, and I'm mostly in aggreement with you. Pretty much everyone involved here was negligent in some way if you ask me. My point was only that I don't think this would set any kind of precedent since it's so far from a simple case of "an owner of a self driving car".

mr_toad 8y ago

> What is not clear is whether or not this task is even humanly feasible Not watching TV while you’re supposed to be driving is a pretty low bar. But apparently, humans can’t even be relied on to do this, even when it’s their one single job.

acdha 8y ago
If you think it’s easy, go sit in the passenger seat without talking or using your phone and see how long you can maintain focus. There are decades of experiments showing that boredom and fatigue are huge problems for this kind of work where someone is mostly idle except for rare events. It’s why the TSA mixes fake images into the stream on X-ray scans so people see things regularly and don’t zone out. Anyone smart asking humans to do repetitive work builds layers of safeguards in to keep people engaged – or in Uber’s case they just hope they can shift blame to the person in a bad design, which worked so far.
- cma 8y ago
  Uber should have had people remote-monitoring the monitors at this early stage.
  - acdha 8y ago
    I saw a different company’s car today: they’re being responsible and have two people in the car.
    - masklinn 8y ago
      Pretty much all of them do. In fact Uber used to do that before they moved to AZ. Having a single person responsible for driving & monitoring was a way to save cost and log more miles.
- cf498 8y ago
  Also >"The system is not designed to alert the operator," the report notes.

usaphp 8y ago

> If I am to serve a 1 year prison sentence for faulty code I wonder why would a programmer go and work on a code when he is always in danger of going to prison for every mistake he makes. We all know that bugs in software happen and it’s impossible to write bug free code

Alex3917 8y ago
> I wonder why would a programmer go and work on a code when he is always in danger of going to prison for every mistake he makes. You don't go to prison for "mistakes", you go to prison for criminal negligence. Given that you can go to prison for criminal negligence in any other profession (e.g. a dentist who exposes thousands of people to HIV), why would programmers be the one category of job that's exempt from criminal laws?
- jdminhbg 8y ago
  > You don't go to prison for "mistakes", you go to prison for criminal negligence. The difference between the two is what a politically-motivated prosecutor can convince a jury of.
  - vkou 8y ago
    If a politically motivated prosecutor can convince a jury of twelve people that Bill killed Dave, by using his psychic brain powers, or, alternatively, by sticking pins into a voodoo doll, or some other form of non-scientific witchcraft, Bill is going to hang. The hypothetical gullibility of juries doesn't mean that we shouldn't have laws against murder, juries, or prosecutors.
    - jdminhbg 8y ago
      The line between mistake and negligence is a lot blurrier than the line between killed and didn’t kill.
SiempreViernes 8y ago
If the programmer can’tbe bothered to put in enough verification effort to avoid being negligent, they should not be in charge of safety critical code. Plainly: don’t pretend there aren’t ways to ensure correctness of systems.

Dylan16807 8y ago

The code wasn't faulty. The configuration was explicitly set to not brake, as far as I understand. If someone got in trouble here, it wouldn't be for programming.

wmeredith 8y ago

I know it’s glib, but just on the face of it, I find the idea of Uber taking responsibility for anything a laughable proposition.

roenxi 8y ago

> If I am to serve a 1 year prison sentence for faulty code or a deprecated LIDAR sensor, I don't see in what scenario I would be willing to leave my fate in the hands of a self-driving car manufacturer. What if we came up with a reasonably objective test of driver skill and the car outperforms you? This is and remains the standard argument - the car doesn't have to be perfect, it just has to beat a human having a bad day. If you are worried about the liability, maybe keep the car serviced and sit in the drivers seat and look out of the window instead of watching a movie. Tell the car to dive slowly maybe. I feel sorry for the driver who is creating this precedent, but this is hardly a road block. The average human is not a great driver.

deleted 8y ago
[deleted]