5 ms·

The fact that it occurs so often should suggest to you that autopilot itself makes people too comfortable and not pay attention.

jerf 5y ago
I, and many others, take the position that there really isn't any practical gradation between more-or-less conventional driving, maybe with light lane switching assistance or something, and full on "Level 5" autonomous driving, because the idea that human brains can watch something for hours at a time only to take instant, sub-second actions every few hours is an insane model of human cognition. I don't disagree with the idea of having a gradient to measure our progress up to full autonomy, but I haven't seen much to say that we should deploy the intermediate stages. Until autonomy can be reasonably guaranteed at at least human levels of safety under the assumption of no human control, it really shouldn't be deployed. "But the human had 673 milliseconds to take control and avert the accident! It's their fault!" is simply not a reasonable defense.
- joelwilliamson 5y ago
  Shouldn't the objection be to anything less than Level 4, rather than Level 5? Level 4 requires that the vehicle can safely stop itself and wait for a human to take over when it is faced with a situation it can't handle. (There's an argument that if you take away the routine parts of driving, humans won't be able to deal with anomalous situations, but that's a different argument than the attentiveness one.)
  - jerf 5y ago
    If that is possible, yes. I'm not convinced it is possible to create an autonomous vehicle that only panics in situations where there is a safe pull-over, or, if such a vehicle exists, that it'll be considered "autonomous" when it can't go past 25 or 35 mph or something. There's plenty of driving situations that may be confusing where pulling over and stopping isn't possible. (And, to be clear, I'm not asking for perfect safety, I'm asking for "human or better", and also understand that there may be some interesting fuzziness near the border where the two have different strengths and weakenesses but come out statistically the same. However, I don't feel like we're there yet.)
galangalalgol 5y ago
Mixed control between automated and human for tasks that require persistent attention is bad. We have known this for decades. The example often given in engineering textbooks is 3 mile island. I suspect that the engineers involved in self driving systems know this too, but the liability is too great to make the switch all at once. It seems better to take baby steps into self driving, but history teaches us it should be all or nothing.
- fallingknife 5y ago
  You could say the same about airplane autopilots, but they have an amazing safety record.
  - kaibee 5y ago
    Airplanes also have 2 pilots and nothing to crash into.
  - Piskvorrr 5y ago
    Far fewer objects to hit in the air: disengagement means "keep going this way," rarely "and crash into an eventual obstacle". Plus, their early safety record is written in blood like everything else is.
    - fallingknife 5y ago
      > Plus, their early safety record is written in blood like everything else is. Yeah, and that's how it's going to be with car autopilots. Which is why it pisses me off that people want to stop progress on it.
      - Piskvorrr 5y ago
        Interesting typoes in "be unwilling test subjects". Feel free to volunteer, though. Elaine Herzberg sure didn't.
  - meepmorp 5y ago

jerf 5y ago

I, and many others, take the position that there really isn't any practical gradation between more-or-less conventional driving, maybe with light lane switching assistance or something, and full on "Level 5" autonomous driving, because the idea that human brains can watch something for hours at a time only to take instant, sub-second actions every few hours is an insane model of human cognition. I don't disagree with the idea of having a gradient to measure our progress up to full autonomy, but I haven't seen much to say that we should deploy the intermediate stages. Until autonomy can be reasonably guaranteed at at least human levels of safety under the assumption of no human control, it really shouldn't be deployed. "But the human had 673 milliseconds to take control and avert the accident! It's their fault!" is simply not a reasonable defense.

joelwilliamson 5y ago
Shouldn't the objection be to anything less than Level 4, rather than Level 5? Level 4 requires that the vehicle can safely stop itself and wait for a human to take over when it is faced with a situation it can't handle. (There's an argument that if you take away the routine parts of driving, humans won't be able to deal with anomalous situations, but that's a different argument than the attentiveness one.)
- jerf 5y ago
  If that is possible, yes. I'm not convinced it is possible to create an autonomous vehicle that only panics in situations where there is a safe pull-over, or, if such a vehicle exists, that it'll be considered "autonomous" when it can't go past 25 or 35 mph or something. There's plenty of driving situations that may be confusing where pulling over and stopping isn't possible. (And, to be clear, I'm not asking for perfect safety, I'm asking for "human or better", and also understand that there may be some interesting fuzziness near the border where the two have different strengths and weakenesses but come out statistically the same. However, I don't feel like we're there yet.)

galangalalgol 5y ago

Mixed control between automated and human for tasks that require persistent attention is bad. We have known this for decades. The example often given in engineering textbooks is 3 mile island. I suspect that the engineers involved in self driving systems know this too, but the liability is too great to make the switch all at once. It seems better to take baby steps into self driving, but history teaches us it should be all or nothing.

fallingknife 5y ago
You could say the same about airplane autopilots, but they have an amazing safety record.
- kaibee 5y ago
  Airplanes also have 2 pilots and nothing to crash into.
- Piskvorrr 5y ago
  Far fewer objects to hit in the air: disengagement means "keep going this way," rarely "and crash into an eventual obstacle". Plus, their early safety record is written in blood like everything else is.
  - fallingknife 5y ago
    > Plus, their early safety record is written in blood like everything else is. Yeah, and that's how it's going to be with car autopilots. Which is why it pisses me off that people want to stop progress on it.
    - Piskvorrr 5y ago
      Interesting typoes in "be unwilling test subjects". Feel free to volunteer, though. Elaine Herzberg sure didn't.
- meepmorp 5y ago