5 ms·

I misread your first sentence, in that having multiple pilots is not necessarily a good thing, like multiple cooks. You have a main and a backup pilot, but eit

I misread your first sentence, in that having multiple pilots is not necessarily a good thing, like multiple cooks. You have a main and a backup pilot, but either one must be 100% capable of doing it on their own. The backup is silently double checking, but their assignments are more about ensuring that the copilot doesn't just check out because they're human. If the copilot ever has to say "don't do that it's going to kill us all" it's a crisis. Lidar is a good backup but the car must be able to with without it. You can't drive just with lidar; it's like driving by Braille. Lidar can't even read a stop light. If they cannot handle it with just the visuals, the car should be not be allowed on the road. I concur that is terrifying that he was allowed to go without the backup that stops it from killing people. Human co drivers are not a good enough backup. But he's also not wrong that the visual system must be practically perfect -- if it's possible at all. Which it surely isn't yet.

Retric 2y ago
Cars unlike aircraft don’t need to move forward to maintain safety. If the lidar normally has very good uptime but happens to break, you need to safely come to a complete stop and that’s basically it. Trusting vision systems for 30 seconds or even 30 minutes over the lifetime of a car is very different than trusting them for 30,000 hours. So for edge cases sure, add a “this is an emergency drive to a hospital without LiDAR” mode but you don’t need to handle normal driving without them.
- larvaetron 2y ago
  > Cars unlike aircraft don’t need to move forward to maintain safety. They do if they're crossing railroad tracks.
  - david38 2y ago
    A narrow case easily accounted for
    - deleted 2y ago
      [deleted]
    - ben_w 2y ago
      Enumerating all the narrow edge cases has proven surprisingly difficult.
      - triplesec 2y ago
        but train tracks is not that kind of edge case, tbh, it's so vanishingly situational
        ben_w 2y ago
        "Train tracks" are a subset of all the edge cases in the set "car can't just stop and stay stopped and that's safe".
  - Retric 2y ago
    You can get out of a car on railroad tracks. However, “safely come to a complete stop” includes turns and freeways which are more serious concerns. That’s why I said 30 seconds / 30 minutes not 3 seconds. The idea is to go to somewhere safe and pull off the road not just slam on the breaks and hope your not on a curve.
  - maximilianthe1 2y ago
    In case of self-drive malfunction you still have a gas pedal under a human's foot
- throw0101d 2y ago
  > So for edge cases sure, add a “this is an emergency drive to a hospital without LiDAR” mode but you don’t need to handle normal driving without them. As a comparison for planes, there are not "this is an emergency, please land yourself" buttons for smaller aircraft: * https://www.garmin.com/en-US/blog/aviation/five-ways-garmin-autoland-offers-peace-of-mind-to-aircraft-pilots-and-passengers/ https://www.garmin.com/en-US/blog/aviation/five-ways-garmin-... * https://www.garmin.com/en-US/legal/aluse/ https://www.garmin.com/en-US/legal/aluse/
  - Retric 2y ago
    Interesting: “Autoland is designed to be used in emergency situations only and should not be used in nonemergency situations where the pilot is fully capable of landing the aircraft.” https://www.garmin.com/en-US/legal/aluse/ https://www.garmin.com/en-US/legal/aluse/
- bee_rider 2y ago
  There are cases where cars do need to move. * train tracks as mentioned by another comment * getting out of the way of emergency vehicles or following instructions from emergency workers I don’t think this detracts from the overall point—if the uptime is very good then the times when LiDAR doesn’t work and the car needs to move might practically never happen. Or the car might be able to move forward to a safe position in some limp-along mode. WRT “this is an emergency drive to a hospital without LiDAR,” I think that would be pretty bad to include. What exactly qualifies as an emergency (this will be abused by some users)? And anyway, in most cases it is better to have an ambulance for an emergency. Finally, an emergency on my part doesn’t entitle me to endanger society. Rather, the types of behavior that the car can perform without all the sensors should be kept track of. If the car can limp along without LiDAR, then that’s something it can do (with the caveat that some roads are not safe to drive far under the speed limit on).
  - Retric 2y ago
    The same basic questions arise for things like a tire blowout. At minimum cars need to handle being on a curve when LiDAR or vision etc cuts off. Stopping on a freeway or railroad is a real risk even if people can get out, but now you’re doing risk mitigation calculations not just trying to handle normal driving. “What if someone is asleep in a level 5 car?” is a question worth considering, but we’re a long way self driving without any from of human backup. The hospital thing is for the very real possibility of being outside of cellphone range, and represents the apex of gracefully shutting down the ride vs refusing to travel while in a less safe condition.
    - bumby 2y ago
      We absolutely should expect safety critical software to mitigate risk. The idea of “welp, the software is confused so just toss it back to a human” is a cop-out in risk mitigation. At the most generous, it constitutes an “administrative” control, which is the least preferred option, tantamount to giving directions in the users manual. The general control hierarchy is: remove the hazard, engineering control, PPE, administrative control.
travisjungroth 2y ago
> You have a main and a backup pilot, but either one must be 100% capable of doing it on their own. The backup is silently double checking, but their assignments are more about ensuring that the copilot doesn't just check out because they're human. This is not how flying works in a multi-crew environment. It’s a common misconception about the dynamic. Both pilots have active roles. Pilots also generally take turns who is manipulating the flight controls (“flying the airplane”) each flight.

Retric 2y ago

Cars unlike aircraft don’t need to move forward to maintain safety. If the lidar normally has very good uptime but happens to break, you need to safely come to a complete stop and that’s basically it. Trusting vision systems for 30 seconds or even 30 minutes over the lifetime of a car is very different than trusting them for 30,000 hours. So for edge cases sure, add a “this is an emergency drive to a hospital without LiDAR” mode but you don’t need to handle normal driving without them.

larvaetron 2y ago
> Cars unlike aircraft don’t need to move forward to maintain safety. They do if they're crossing railroad tracks.
- david38 2y ago
  A narrow case easily accounted for
  - deleted 2y ago
    [deleted]
  - ben_w 2y ago
    Enumerating all the narrow edge cases has proven surprisingly difficult.
    - triplesec 2y ago
      but train tracks is not that kind of edge case, tbh, it's so vanishingly situational
      - ben_w 2y ago
        "Train tracks" are a subset of all the edge cases in the set "car can't just stop and stay stopped and that's safe".
- Retric 2y ago
  You can get out of a car on railroad tracks. However, “safely come to a complete stop” includes turns and freeways which are more serious concerns. That’s why I said 30 seconds / 30 minutes not 3 seconds. The idea is to go to somewhere safe and pull off the road not just slam on the breaks and hope your not on a curve.
- maximilianthe1 2y ago
  In case of self-drive malfunction you still have a gas pedal under a human's foot
throw0101d 2y ago
> So for edge cases sure, add a “this is an emergency drive to a hospital without LiDAR” mode but you don’t need to handle normal driving without them. As a comparison for planes, there are not "this is an emergency, please land yourself" buttons for smaller aircraft: * https://www.garmin.com/en-US/blog/aviation/five-ways-garmin-autoland-offers-peace-of-mind-to-aircraft-pilots-and-passengers/ https://www.garmin.com/en-US/blog/aviation/five-ways-garmin-... * https://www.garmin.com/en-US/legal/aluse/ https://www.garmin.com/en-US/legal/aluse/
- Retric 2y ago
  Interesting: “Autoland is designed to be used in emergency situations only and should not be used in nonemergency situations where the pilot is fully capable of landing the aircraft.” https://www.garmin.com/en-US/legal/aluse/ https://www.garmin.com/en-US/legal/aluse/
bee_rider 2y ago
There are cases where cars do need to move. * train tracks as mentioned by another comment * getting out of the way of emergency vehicles or following instructions from emergency workers I don’t think this detracts from the overall point—if the uptime is very good then the times when LiDAR doesn’t work and the car needs to move might practically never happen. Or the car might be able to move forward to a safe position in some limp-along mode. WRT “this is an emergency drive to a hospital without LiDAR,” I think that would be pretty bad to include. What exactly qualifies as an emergency (this will be abused by some users)? And anyway, in most cases it is better to have an ambulance for an emergency. Finally, an emergency on my part doesn’t entitle me to endanger society. Rather, the types of behavior that the car can perform without all the sensors should be kept track of. If the car can limp along without LiDAR, then that’s something it can do (with the caveat that some roads are not safe to drive far under the speed limit on).
- Retric 2y ago
  The same basic questions arise for things like a tire blowout. At minimum cars need to handle being on a curve when LiDAR or vision etc cuts off. Stopping on a freeway or railroad is a real risk even if people can get out, but now you’re doing risk mitigation calculations not just trying to handle normal driving. “What if someone is asleep in a level 5 car?” is a question worth considering, but we’re a long way self driving without any from of human backup. The hospital thing is for the very real possibility of being outside of cellphone range, and represents the apex of gracefully shutting down the ride vs refusing to travel while in a less safe condition.
  - bumby 2y ago
    We absolutely should expect safety critical software to mitigate risk. The idea of “welp, the software is confused so just toss it back to a human” is a cop-out in risk mitigation. At the most generous, it constitutes an “administrative” control, which is the least preferred option, tantamount to giving directions in the users manual. The general control hierarchy is: remove the hazard, engineering control, PPE, administrative control.

travisjungroth 2y ago

> You have a main and a backup pilot, but either one must be 100% capable of doing it on their own. The backup is silently double checking, but their assignments are more about ensuring that the copilot doesn't just check out because they're human. This is not how flying works in a multi-crew environment. It’s a common misconception about the dynamic. Both pilots have active roles. Pilots also generally take turns who is manipulating the flight controls (“flying the airplane”) each flight.