4 ms·

So if an automaker can make a system that is capable of handling any driving task safely but occasionally ran itself off the road on purpose for no reason, as l

by MBCook 4y ago

So if an automaker can make a system that is capable of handling any driving task safely but occasionally ran itself off the road on purpose for no reason, as long as it’s a net positive in lives saved it should be allowed? I don’t agree with that assessment. It needs to make fewer mistakes than humans at all times, not just most of the time. Great with a disaster here and there is terrible.

epivosism 4y ago
I see where you're going, but how do you actually justify your choices? A: occasional random suicide (100 deaths/year), no other deaths B: no random suicide, 10k deaths per year from normal driving You would be saying you choose B? It's not really about the exact numbers here - just trying to get on the same page - would you really weigh your uncomfortableness with "random deaths" highly enough that you would accept increased actual deaths? Another possibility is that the random death situation really is a signal of "actually hard-to-detect badness". If so, it should come out in statistics and we should easily be able to come to agreement in that case. But, if the choice really is between accepting "weird mistakes+lower deaths" or "status quo+many deaths" I feel pretty uncomfortable accepting the latter knowing the real-world consequences.
- MBCook 4y ago
  This is a human comfort thing, not a reason thing. As someone else in this discussion said if computers do something that looks stupid to people, they won’t trust it. What if the investigation here showed the Tesla was 100% correct to do what it did? What it did is so odd I wouldn’t want to trust it. Maybe the problem is it needs to fail in a more human-like way? If the death numbers get low enough, I think it will likely speak for itself and people will trust it. But it can’t be a 30% reduction in deaths. It needs to be like 95%. Enough that it feels safer instead of questionable/dangerous. Even if it was actually safer the entire time.
stkdump 4y ago
I heard an interesting philosophical discussion many years back, where it was stated that we are morally obligated to outlaw non-self-driving cars as soon as it is proven that self-driving cars kill fewer people. It is not so easy to argue against that, because if you do, you rate your own right to choose higher than another persons right to live.
abigail95 4y ago
Firstly, human error in at least 7 out of the 8 cars caused this. FSD didn't take control away from the human, it encountered something outside of it's limits and returned full control to the human. Secondly, that's what planes are. A lot of new plane safety comes from very automated, very complicated systems. In some situations these planes fail in a way that leaves the pilot in a more dangerous plane than one with no electronics at all. Why do we tolerate this? Because it doesn't matter how poorly something handles failure, what dominates is the base rate of the failure occuring. If you ban these extra-automated driver aids, and people drive Teslas less, more people die. Same as if it causes a delay in developing real FSD. I would rather live next door to an RBMK-1000, than drive a car 10 years older than my current one. I caution you against changing automation policy based on an accident caused by human error.
- rini17 4y ago
  So you categorize the 737MAX accidents as human errors too and thus they should not affect policy?
  - sidibe 4y ago
    That's a very different situation. Namely, I don't own Boeing stock.
- SyzygistSix 4y ago
  My thought was that the cars following needed to have adaptive cruise control engaged and perhaps then the accidents would be avoided.
- MBCook 4y ago
  > FSD didn't take control away from the human, it encountered something outside of it's limits and returned full control to the human. We don’t know if that’s what happened.

by MBCook 4y ago

epivosism 4y ago

I see where you're going, but how do you actually justify your choices? A: occasional random suicide (100 deaths/year), no other deaths B: no random suicide, 10k deaths per year from normal driving You would be saying you choose B? It's not really about the exact numbers here - just trying to get on the same page - would you really weigh your uncomfortableness with "random deaths" highly enough that you would accept increased actual deaths? Another possibility is that the random death situation really is a signal of "actually hard-to-detect badness". If so, it should come out in statistics and we should easily be able to come to agreement in that case. But, if the choice really is between accepting "weird mistakes+lower deaths" or "status quo+many deaths" I feel pretty uncomfortable accepting the latter knowing the real-world consequences.

MBCook 4y ago
This is a human comfort thing, not a reason thing. As someone else in this discussion said if computers do something that looks stupid to people, they won’t trust it. What if the investigation here showed the Tesla was 100% correct to do what it did? What it did is so odd I wouldn’t want to trust it. Maybe the problem is it needs to fail in a more human-like way? If the death numbers get low enough, I think it will likely speak for itself and people will trust it. But it can’t be a 30% reduction in deaths. It needs to be like 95%. Enough that it feels safer instead of questionable/dangerous. Even if it was actually safer the entire time.

stkdump 4y ago

I heard an interesting philosophical discussion many years back, where it was stated that we are morally obligated to outlaw non-self-driving cars as soon as it is proven that self-driving cars kill fewer people. It is not so easy to argue against that, because if you do, you rate your own right to choose higher than another persons right to live.

abigail95 4y ago

Firstly, human error in at least 7 out of the 8 cars caused this. FSD didn't take control away from the human, it encountered something outside of it's limits and returned full control to the human. Secondly, that's what planes are. A lot of new plane safety comes from very automated, very complicated systems. In some situations these planes fail in a way that leaves the pilot in a more dangerous plane than one with no electronics at all. Why do we tolerate this? Because it doesn't matter how poorly something handles failure, what dominates is the base rate of the failure occuring. If you ban these extra-automated driver aids, and people drive Teslas less, more people die. Same as if it causes a delay in developing real FSD. I would rather live next door to an RBMK-1000, than drive a car 10 years older than my current one. I caution you against changing automation policy based on an accident caused by human error.

rini17 4y ago
So you categorize the 737MAX accidents as human errors too and thus they should not affect policy?
- sidibe 4y ago
  That's a very different situation. Namely, I don't own Boeing stock.
SyzygistSix 4y ago
My thought was that the cars following needed to have adaptive cruise control engaged and perhaps then the accidents would be avoided.
MBCook 4y ago
> FSD didn't take control away from the human, it encountered something outside of it's limits and returned full control to the human. We don’t know if that’s what happened.