6 ms·

These spectacular fails weaken the rationalist's arguments that "as long as FSD achieves lower deaths per km driven (or any other metric) than humans" then FSD

by j7ake 5y ago

These spectacular fails weaken the rationalist's arguments that "as long as FSD achieves lower deaths per km driven (or any other metric) than humans" then FSD should be accepted in favor of human driving. Even if "on aggregate" FSD performs safer (by some metric) than humans, as long as FSD continues to fail in a way that would have been easily preventable had a human been at the wheel, FSD will not be accepted into society.

manmal 5y ago
I think this is already true though for highway driving. Highways are long and tiring, and the surroundings model is easy to get right, so computers have an advantage. Most manufacturers offer a usable cruise control which is safe and can probably be active 90% of the time spent on highways. I often switch it on in my 3yo Hyundai as an extra safety measure in case the car in front of me unexpectedly brakes while I‘m not looking there. Add to that a lane keeping assistant and lane change assistant, and you don‘t need to do much. Except for when the radar doesn’t see an obstacle in front of you, eg because the car in front of you just changed lanes. That needs to be looked out for.
- deleted 5y ago
  [deleted]
cptskippy 5y ago
> spectacular There was nothing spectacular about those failures, I would say because the driver was attentive and caught/corrected the car. That's not to say some of these fails could not have ended in catastrophe, but to call them spectacular is quite the exaggeration. One of those "spectacular fails" was displaying two stop signs in the UI on top of each other while properly treating it as one stop. Using hyperbole like this only makes people ignore or dismiss your otherwise valid point.
cogman10 5y ago
Yeah, I've brought this point up in other locations. It does not matter that any autonomous driving tech is safer than human drivers. They MUST be perfect for the general public to accept them. The only accidents they'd be allowed to get into are ones that are beyond their control. Algorithmic accidents, no matter how rare they are, won't be tolerated by the general public. Nobody will accept a self driving car running over a cat or rear ending a bus even if regular humans do that all day long. The expectation for self driving cars is a perfectly attentive driver making correct decisions. Because, that's what you theoretically have. The computer's mind doesn't "wander" and it can't be distracted. There's no excuse for it to drive worse than the best human driver.
- j7ake 5y ago
  Imagine if algorithmic accidents had biases. For example, let's say a car tended to crash into children (maybe they are harder to detect with cameras), more often than adults. This type of algorithmic bias would be unacceptable no matter how safe FSD were on aggregate. So you're right, the only bar to reach is perfection (which is impossible), because algorithmic errors have biases that will likely deviate from human biases.
  - cogman10 5y ago
    Call me an optimist, but I don't think it's impossible. That said, there are going to be a lot of dead small animals due to autonomous vehicles. I'd hope that whoever develops the system has some good training data to stop it from hitting children. The issue will be that it's going to be real hard to make a system that can tell the difference between a plastic bag and a poodle.
  - truffdog 5y ago
    > let's say a car tended to crash into children (maybe they are harder to detect with cameras), more often than adults This is already true today of human drivers because of the tall SUVs that are so popular. Do you think matching biases will be acceptable?
    - cogman10 5y ago
      Would you accept it? Would you be ok if a car without a driver ran over your kid, even if they were playing in the local street? I'd say, absolutely not. The only way we'd accept that is if the kid darted out before the vehicle could slow down, and even then we'd expect super human braking to (hopefully) avoid serious injury. Also, self driving cars have and advantage that they can put cameras in places typical drivers eyes aren't. They should be able to see a lot more than you can from the driver's seat.
    - ggreer 5y ago
      That's not true. First, the vast majority of pedestrian deaths are adults. In 2018, a total of 206 age 15 or younger were killed by cars. Compare that to 5,965 killed who were age 16 or older.[1] Both in absolute numbers and relative to population, children are far less likely to be run over and killed than adults. Second, while light trucks (vans, SUVs, & pickups) are 1.45x more deadly to pedestrians than cars, buses are far more dangerous than either. Motorcycles (which have excellent visibility) are particularly deadly to child pedestrians. From United States pedestrian fatality rates by vehicle type[2]: > Compared with cars, the RR of killing a pedestrian per vehicle mile was 7.97 (95% CI 6.33 to 10.04) for buses; 1.93 (95% CI 1.30 to 2.86) for motorcycles; 1.45 (95% CI 1.37 to 1.55) for light trucks, and 0.96 (95% CI 0.79 to 1.18) for heavy trucks. Compared with cars, buses were 11.85 times (95% CI 6.07 to 23.12) and motorcycles were 3.77 times (95% CI 1.40 to 10.20) more likely per mile to kill children 0–14 years old. Buses were 16.70 times (95% CI 7.30 to 38.19) more likely to kill adults age 85 or older than were cars. The risk of killing a pedestrian per vehicle mile traveled in an urban area was 1.57 times (95% CI 1.47 to 1.67) the risk in a rural area. All else equal, being hit by a larger vehicle does increase the risk of severe injury or death, but all else isn't equal. Larger vehicles tend to be more visible, louder, and slower than their smaller counterparts. Different types of vehicles are driven in different environments with different propensities for mingling with pedestrians. If vehicle mass and blind spots were the main factors in pedestrian deaths, we should have seen deaths skyrocket over the past 40 years (as cars got bigger and bulkier for greater passenger safety). Instead we saw pedestrian deaths decrease. 1. https://docs.google.com/spreadsheets/d/e/2PACX-1vRqGqodKkWkSLY3iSiP_js0zovJxnW8MEIvdcr6wfT2KF_FhoQO2MGklMo4VZCuODAy2gd6S1_2Mev6/pubhtml# https://docs.google.com/spreadsheets/d/e/2PACX-1vRqGqodKkWkS... 2. https://injuryprevention.bmj.com/content/11/4/232 https://injuryprevention.bmj.com/content/11/4/232
the8472 5y ago
I think you misunderstand the argument. It is that if, hypothetically, FSD really did save human lives on average then it should be accepted as the default mode of driving. It would be a net win in human lives after all. But the "should" can also acknowledge that people irrationally won't accept this net life-saving technology because it will redistribute the deaths in ways which they're not accustomed to. So it's as much a statement about utility as a statement about the need to convince people. Of course this is all theoretical. If we had solid evidence that it performs better than humans in some scenarios but worse in others then we could save even more lives by only allowing it to run in those cases where it does and only do shadow-mode piloting in the others (or those who opt into lab rat mode). Enabling it by default only makes sense if we do know that it performs better on average and we do not know when it does.
- paxys 5y ago
  I don't agree with the former argument either. I'm not going to accept a self driving system unless it increases my personal safety. If the system doubles my accident rate but cuts that of drunks by a factor of 10 (thus improving the national average), it isn't irrational to not want it for myself.
  - the8472 5y ago
    > The car has to be better at driving than me But you can also be a pedestrian or passenger. Do you not want everyone else to be less likely to kill you? Also, should you really trust your own estimate of your driving safety? > McCormick, Walkey and Green (1986) found similar results in their study, asking 178 participants to evaluate their position on eight different dimensions of driving skills (examples include the "dangerous–safe" dimension and the "considerate–inconsiderate" dimension). Only a small minority rated themselves as below the median, and when all eight dimensions were considered together it was found that almost 80% of participants had evaluated themselves as being an above-average driver.[30]
  - aeternum 5y ago
    Why do you believe the system is likely to double your accident rate? Statistically autopilot gets in fewer accidents than the average human driver. And if you believe you are an above average driver, why do you believe you would be unable to intervene if needed?

by j7ake 5y ago

manmal 5y ago

I think this is already true though for highway driving. Highways are long and tiring, and the surroundings model is easy to get right, so computers have an advantage. Most manufacturers offer a usable cruise control which is safe and can probably be active 90% of the time spent on highways. I often switch it on in my 3yo Hyundai as an extra safety measure in case the car in front of me unexpectedly brakes while I‘m not looking there. Add to that a lane keeping assistant and lane change assistant, and you don‘t need to do much. Except for when the radar doesn’t see an obstacle in front of you, eg because the car in front of you just changed lanes. That needs to be looked out for.

deleted 5y ago
[deleted]

cptskippy 5y ago

> spectacular There was nothing spectacular about those failures, I would say because the driver was attentive and caught/corrected the car. That's not to say some of these fails could not have ended in catastrophe, but to call them spectacular is quite the exaggeration. One of those "spectacular fails" was displaying two stop signs in the UI on top of each other while properly treating it as one stop. Using hyperbole like this only makes people ignore or dismiss your otherwise valid point.

cogman10 5y ago

Yeah, I've brought this point up in other locations. It does not matter that any autonomous driving tech is safer than human drivers. They MUST be perfect for the general public to accept them. The only accidents they'd be allowed to get into are ones that are beyond their control. Algorithmic accidents, no matter how rare they are, won't be tolerated by the general public. Nobody will accept a self driving car running over a cat or rear ending a bus even if regular humans do that all day long. The expectation for self driving cars is a perfectly attentive driver making correct decisions. Because, that's what you theoretically have. The computer's mind doesn't "wander" and it can't be distracted. There's no excuse for it to drive worse than the best human driver.

j7ake 5y ago
Imagine if algorithmic accidents had biases. For example, let's say a car tended to crash into children (maybe they are harder to detect with cameras), more often than adults. This type of algorithmic bias would be unacceptable no matter how safe FSD were on aggregate. So you're right, the only bar to reach is perfection (which is impossible), because algorithmic errors have biases that will likely deviate from human biases.
- cogman10 5y ago
  Call me an optimist, but I don't think it's impossible. That said, there are going to be a lot of dead small animals due to autonomous vehicles. I'd hope that whoever develops the system has some good training data to stop it from hitting children. The issue will be that it's going to be real hard to make a system that can tell the difference between a plastic bag and a poodle.
- truffdog 5y ago
  > let's say a car tended to crash into children (maybe they are harder to detect with cameras), more often than adults This is already true today of human drivers because of the tall SUVs that are so popular. Do you think matching biases will be acceptable?
  - cogman10 5y ago
    Would you accept it? Would you be ok if a car without a driver ran over your kid, even if they were playing in the local street? I'd say, absolutely not. The only way we'd accept that is if the kid darted out before the vehicle could slow down, and even then we'd expect super human braking to (hopefully) avoid serious injury. Also, self driving cars have and advantage that they can put cameras in places typical drivers eyes aren't. They should be able to see a lot more than you can from the driver's seat.
  - ggreer 5y ago
    That's not true. First, the vast majority of pedestrian deaths are adults. In 2018, a total of 206 age 15 or younger were killed by cars. Compare that to 5,965 killed who were age 16 or older.[1] Both in absolute numbers and relative to population, children are far less likely to be run over and killed than adults. Second, while light trucks (vans, SUVs, & pickups) are 1.45x more deadly to pedestrians than cars, buses are far more dangerous than either. Motorcycles (which have excellent visibility) are particularly deadly to child pedestrians. From United States pedestrian fatality rates by vehicle type[2]: > Compared with cars, the RR of killing a pedestrian per vehicle mile was 7.97 (95% CI 6.33 to 10.04) for buses; 1.93 (95% CI 1.30 to 2.86) for motorcycles; 1.45 (95% CI 1.37 to 1.55) for light trucks, and 0.96 (95% CI 0.79 to 1.18) for heavy trucks. Compared with cars, buses were 11.85 times (95% CI 6.07 to 23.12) and motorcycles were 3.77 times (95% CI 1.40 to 10.20) more likely per mile to kill children 0–14 years old. Buses were 16.70 times (95% CI 7.30 to 38.19) more likely to kill adults age 85 or older than were cars. The risk of killing a pedestrian per vehicle mile traveled in an urban area was 1.57 times (95% CI 1.47 to 1.67) the risk in a rural area. All else equal, being hit by a larger vehicle does increase the risk of severe injury or death, but all else isn't equal. Larger vehicles tend to be more visible, louder, and slower than their smaller counterparts. Different types of vehicles are driven in different environments with different propensities for mingling with pedestrians. If vehicle mass and blind spots were the main factors in pedestrian deaths, we should have seen deaths skyrocket over the past 40 years (as cars got bigger and bulkier for greater passenger safety). Instead we saw pedestrian deaths decrease. 1. https://docs.google.com/spreadsheets/d/e/2PACX-1vRqGqodKkWkSLY3iSiP_js0zovJxnW8MEIvdcr6wfT2KF_FhoQO2MGklMo4VZCuODAy2gd6S1_2Mev6/pubhtml# https://docs.google.com/spreadsheets/d/e/2PACX-1vRqGqodKkWkS... 2. https://injuryprevention.bmj.com/content/11/4/232 https://injuryprevention.bmj.com/content/11/4/232

the8472 5y ago

I think you misunderstand the argument. It is that if, hypothetically, FSD really did save human lives on average then it should be accepted as the default mode of driving. It would be a net win in human lives after all. But the "should" can also acknowledge that people irrationally won't accept this net life-saving technology because it will redistribute the deaths in ways which they're not accustomed to. So it's as much a statement about utility as a statement about the need to convince people. Of course this is all theoretical. If we had solid evidence that it performs better than humans in some scenarios but worse in others then we could save even more lives by only allowing it to run in those cases where it does and only do shadow-mode piloting in the others (or those who opt into lab rat mode). Enabling it by default only makes sense if we do know that it performs better on average and we do not know when it does.

paxys 5y ago
I don't agree with the former argument either. I'm not going to accept a self driving system unless it increases my personal safety. If the system doubles my accident rate but cuts that of drunks by a factor of 10 (thus improving the national average), it isn't irrational to not want it for myself.
- the8472 5y ago
  > The car has to be better at driving than me But you can also be a pedestrian or passenger. Do you not want everyone else to be less likely to kill you? Also, should you really trust your own estimate of your driving safety? > McCormick, Walkey and Green (1986) found similar results in their study, asking 178 participants to evaluate their position on eight different dimensions of driving skills (examples include the "dangerous–safe" dimension and the "considerate–inconsiderate" dimension). Only a small minority rated themselves as below the median, and when all eight dimensions were considered together it was found that almost 80% of participants had evaluated themselves as being an above-average driver.[30]
- aeternum 5y ago
  Why do you believe the system is likely to double your accident rate? Statistically autopilot gets in fewer accidents than the average human driver. And if you believe you are an above average driver, why do you believe you would be unable to intervene if needed?