5 ms·

It depends. If it's an issue with just that vehicle then yes, you don't ground the whole fleet. If it's non-safety-critical, then it needs to be fixed asap but

It depends. If it's an issue with just that vehicle then yes, you don't ground the whole fleet. If it's non-safety-critical, then it needs to be fixed asap but nothing gets grounded. If it's safety critical, then yes, all planes are grounded. But, I'd argue FAA levels of safety aren't suitable for cars. If a car has an engine failure or even a structural failure, they can usually just stop. Planes don't have that option.

logifail 5y ago
> I'd argue FAA levels of safety aren't suitable for cars. If a car has an engine failure or even a structural failure, they can usually just stop. Planes don't have that option. I had supposed we were thinking primarily about software faults and specifically those relating to the self-driving bit of a self-driving car. Best case: self-driving car gets lost and is found with flat battery miles from "home" (?) Worst-case: self-driving car kills cyclist/pedestrian and doesn't even notice or stop. We should spend some thought on how to handle the latter because it will happen sooner or later. To be blunt: if the algo kills, why wouldn't you ground the algo (all instances of it)?
- eru 5y ago
  I wonder whether that would just lead to people using slightly tweaked variants of the same algorithm branded to be different? (Of course, you don't want to fork too much, because there's probably a certification per algorithm needed. But it might make sense for Google to officially field a thousand 'different' algorithms, so that a single incident doesn't ground the entire fleet?)
  - ocdtrekkie 5y ago
    Engineers often fail to realize you can't cheese a judge and if you try to, the law will come down much, much harder on you. Either one of two things will be the case with this approach: 1. The algorithm will be too imperceptibly different to be treated as a different program. Adding some tiny decimals somewhere won't make it unlikely to make the same mistakes, and the entire class of algorithms will be treated as one. 2. The algorithm will be different enough to make different decisions, which means one program is probably "better" than the other(s) and they'll be taking a bigger risk by deploying cars to the road that are more dangerous than others.
    - eru 5y ago
      Judges are human indeed. But things like tax dodges still exist and other loopholes. You just need good enough lawyers to structure things for you. Your second point would lead to a ban on all but one self-driving car systems, when taken seriously?
      - ocdtrekkie 5y ago
        I don't know about that. I do think that if, say, Google operated a spectrum of 1,000 self-driving algorithms, and knew that some were safer than others, a judge would take a very dim view to knowingly deploying less safe cars than others. And even from a business standpoint, it wouldn't be good business to have less safe (aka, higher liability) cars on the road.
        eru 5y ago
        Why would Google know that some of them are less safe than others? Just have variations that don't dominate each other. Variant A could be better at some things, and variant B could be better at other things, etc. (Otherwise, we should use your argument to only allow exactly one company with the 'best' system on the road. Instead of allowing competition.) Google could even officially outsource the different variants to different nominally independent companies, if that's what's legally required to not get the whole fleet grounded.
- snovv_crash 5y ago
  > To be blunt: if the algo kills, why wouldn't you ground the algo (all instances of it)? It depends what the alternative is. If it crashes less than humans, then you keep running it despite the flaws.
- colinmhayes 5y ago
  > if the algo kills, why wouldn't you ground the algo (all instances of it)? Well 40,000 people in the U.S. die in car crashes every year currently, why do we let people drive?

logifail 5y ago

> I'd argue FAA levels of safety aren't suitable for cars. If a car has an engine failure or even a structural failure, they can usually just stop. Planes don't have that option. I had supposed we were thinking primarily about software faults and specifically those relating to the self-driving bit of a self-driving car. Best case: self-driving car gets lost and is found with flat battery miles from "home" (?) Worst-case: self-driving car kills cyclist/pedestrian and doesn't even notice or stop. We should spend some thought on how to handle the latter because it will happen sooner or later. To be blunt: if the algo kills, why wouldn't you ground the algo (all instances of it)?

eru 5y ago
I wonder whether that would just lead to people using slightly tweaked variants of the same algorithm branded to be different? (Of course, you don't want to fork too much, because there's probably a certification per algorithm needed. But it might make sense for Google to officially field a thousand 'different' algorithms, so that a single incident doesn't ground the entire fleet?)
- ocdtrekkie 5y ago
  Engineers often fail to realize you can't cheese a judge and if you try to, the law will come down much, much harder on you. Either one of two things will be the case with this approach: 1. The algorithm will be too imperceptibly different to be treated as a different program. Adding some tiny decimals somewhere won't make it unlikely to make the same mistakes, and the entire class of algorithms will be treated as one. 2. The algorithm will be different enough to make different decisions, which means one program is probably "better" than the other(s) and they'll be taking a bigger risk by deploying cars to the road that are more dangerous than others.
  - eru 5y ago
    Judges are human indeed. But things like tax dodges still exist and other loopholes. You just need good enough lawyers to structure things for you. Your second point would lead to a ban on all but one self-driving car systems, when taken seriously?
    - ocdtrekkie 5y ago
      I don't know about that. I do think that if, say, Google operated a spectrum of 1,000 self-driving algorithms, and knew that some were safer than others, a judge would take a very dim view to knowingly deploying less safe cars than others. And even from a business standpoint, it wouldn't be good business to have less safe (aka, higher liability) cars on the road.
      - eru 5y ago
        Why would Google know that some of them are less safe than others? Just have variations that don't dominate each other. Variant A could be better at some things, and variant B could be better at other things, etc. (Otherwise, we should use your argument to only allow exactly one company with the 'best' system on the road. Instead of allowing competition.) Google could even officially outsource the different variants to different nominally independent companies, if that's what's legally required to not get the whole fleet grounded.
snovv_crash 5y ago
> To be blunt: if the algo kills, why wouldn't you ground the algo (all instances of it)? It depends what the alternative is. If it crashes less than humans, then you keep running it despite the flaws.
colinmhayes 5y ago
> if the algo kills, why wouldn't you ground the algo (all instances of it)? Well 40,000 people in the U.S. die in car crashes every year currently, why do we let people drive?