6 ms·

As a private pilot it astounds me how the self-driving car industry has seemingly taken every lesson we've learned from automation in aviation and chucked it ou

As a private pilot it astounds me how the self-driving car industry has seemingly taken every lesson we've learned from automation in aviation and chucked it out the window in these systems. There's a "trust but verify" element of airplane automation that is ingrained in you as a pilot. The ML aspect of self-driving cars generally make it a lot less easy to reason about and predict. That's problem 1. Problem 2 is then speed; Even in the airline world, there are published minimums at which pilots need to be absolute certain that the AP is doing the right thing before they commit to land, even in cases where the autopilot is responsible for the entire landing (Autoland is a capability that a lot of airliners have had for many decades). What we're talking about here is several seconds heads-up with visual AND instrument confirmation, that the computers are doing what they're supposed to before we allow the plane to land. Not only that, but we bend over backwards ensuring that there is all kinds of redundancy and procedures in place to ensure the accuracy of the system: Multiple instruments using different kinds of technologies confirming one another, greater separation between aircraft to guarantee ILS radio beacon accuracy, etc. Conversely, in the self-driving world, we're having these ML algorithms make split-second decisions with no real way of informing the driver what it is basing it on in an environment that is far less predictable and constantly-changing with someone who is not a professional behind the wheel.

ozim 7y ago
Self driving car approach has to be different because whole idea is different than airplane automation. You want self driving car to be autonomus without need to rely on trained and disciplined human element. End goal is totally different and you cannot make progress to have fully autonomous car if you would come from: "yeah human will be in control all the time, not being stupid or drunk".
- nradov 7y ago
  It's by no means clear that the end goal and approach should be different. Personally I think that aiming toward level 4-5 autonomous vehicles right now is misguided and unlikely to succeed (outside of a few controlled access roads in limited areas). A better approach would be to require the human driver to actively control the vehicle at all times, but have the computer step in to brake or steer when it detects a high probability of a dangerous situation. Like how auto GCAS works on modern aircraft.
  - MR4D 7y ago
    This, completely. A few months ago, I lightly rear-ended a vehicle ahead of me when they stopped at a green light while turning. Legally, I was in the wrong, and it was a stupid mistake. I had looked left to make sure no traffic was coming at the exact moment the person ahead of me stopped. Had either event (my head turning or his breaking) been different by about a second, no accident would have happened. A simple anti-collision system would have prevented this, and saved me some money (and him from some scratches on his bumper). As I have an older car - 2012 model - so this feature wasn't prevalent when I bought it. But my next car will have this capability, if for no other reason than I hate stupid mistakes. This one feature would make my drive to/from work much less stressful (there are a ton of people who will switch lanes in front of you at the last minute and then hit the brakes here in Houston). Add in lane-change warnings, and I think a significant number of accidents could be avoided. They may be small accidents, but a bumper replacement or painting (times 2 - one for each car involved) is easily $1,000, if not more.
    - perl4ever 7y ago
      If you get used to driving a car that actually does this sort of thing well, then the first time you get in a car where it malfunctions or doesn't have the feature, you will have a bad accident because your reflexes will be missing. I got a car that automatically closes the door so you don't have to slam it. Now I keep forgetting to close the door on a normal car properly. It's eliminated my muscle memory in just a few months. Another instance was when I got a car without a clutch interlock. I never would have realized how dependent I was on that until I started it up in gear.
      - MR4D 7y ago
        I agree 100%. To me, I think two possible outcomes will prevail: 1 - All cars add this feature either through regulation or other pressure. 2 - Most cars add this feature, and those that don't see higher accident rates, and therefore higher insurance bills. That should result in those cars being less desirable (and therefore less profitable for the car companies), OR the car companies figure it out and add the missing feature to the rest of their cars. Either way, I think we'll end up there, just not sure of what path we'll go down.
tlb 7y ago
I'm not sure what you're proposing instead. If you built self-driving cars that required pilot-level skills and training to operate them, who would use them?
- wil421 7y ago
  The main point He is trying to make is the AP has several redundant systems and 2-3 humans double checking the instruments and AP. With self driving cars we use some fancy ML/AI where we don’t exactly know why the machine chose to do what it did.
  - cblades 7y ago
    Those aren't comparable things though. Our driver training just flat-out isn't robust enough to support a similar system, and there's only one person in control of the car. He's also leaving out checklist-driven procedure, which is a huge component of airplane safety that is entirely absent from car driving.
    - base698 7y ago
      A scan where you verify what the computer is telling you and cross checking it with what you're seeing could be. Maybe they have a UI showing the path the computer is taking and bounding boxes of all the potential obstacles overlayed on the screen. If there is a disagreement the human takes over. Similar to how if on an instrument approach the altitude doesn't agree with the vertical guidance. You can check other systems to see which instrument has failed and whether it's the vertical guidance or altimeter. Of course, this method of driving and having to pay more attention doesn't jive with the fantasy of getting wasted and having a computer take you home.
      - magduf 7y ago
        >Of course, this method of driving and having to pay more attention doesn't jive with the fantasy of getting wasted and having a computer take you home. Which is why the system you're describing will never come to cars: American drivers would never use it correctly. If it isn't something that complete morons can jump in and use with zero training, it's just not going to work here. What you're describing requires a competent pilot with training, and that's a completely unrealistic expectation for a car in America.
jhayward 7y ago
It's difficult to see what your main point is, but what I take from it is the idea that aircraft autopilots and self-driving cars are two completely separate problem domains, with very different requirements, resources, and operating regimes. It would seem to be foolish to have a goal of having "no fatalities per 100 billion miles", or whatever the current spec level for airline travel is, as the minimum operating requirement for automobiles. Likewise requiring two expert-level operators at the controls, with re-certification every six months seems to be out of scope. So why should the AP try to be designed assuming those things are true?
kazinator 7y ago
Airplanes don't have to react to a three-year-old on a tricycle suddenly emerging from between parked cars.
JamesBarney 7y ago
Autonomous car's don't have to be nearly as safe as airplanes because existing cars are orders of magnitude less safe than airplanes.
- jandrese 7y ago
  Isn't that one of the problems autonomous cars are trying to solve?
  - stcredzero 7y ago
    Would just one order of magnitude improvement count as "solved?"
  - kemitche 7y ago
    The point is that cars don't need to make an immediate jump from "current levels of safety" to "plane levels of safety." Anything that moves the needle noticeably closer to "plane levels" is a solid improvement in terms of reducing unnecessary deaths.
    - perl4ever 7y ago
      Why do you think that autonomous cars will start by replacing the worst drivers, rather than replacing professional drivers, who are much better than the average person? Isn't the biggest potential initial demand for autonomous cars people who don't own cars and currently must choose between bus and taxi?
- LoSboccacc 7y ago
  citation needed. notice how incident per mile statistic where quickly submerged by Tesla around their fourth autopilot death. also note how autopilot are currently d enabled in the less risky part of the journey. also notice how autopilot stat in most articles are compared toward national averages, which include older car with less safety features and bikes.
  - 7y ago
nostrademons 7y ago
It's different because the market for (and culture of) pilots is dramatically different from the market for drivers. I have never met a pilot that didn't religiously follow their checklists. If there is ever any doubt as to whether something is safe, there is no doubt: it is not. Weight limits, weather reports, etc. all are carefully attended to. Meanwhile, 30% of adults sitting in the back seat of a car forget to buckle their seatbelts. About 5% admitted to driving drunk within the last month. I see people on their cell phones all their time. I've ridden with a young woman who was driving with her knees while doing her hair in the rearview mirror (yes, I was freaked out, and no, I never rode with her again). Pilots are professionals, and they act professionally, and they respect the possibility of death every time they get into their vehicle. Drivers are not. There's a long history of every safety innovation in cars actually making accident rates go up, because drivers start taking them for granted and then behaving in ways that compensate for the increased safety and maintain the same level of subjective risk (like driving faster, driving more, or becoming more distracted). The only way to break that loop is to take the human out of driver's seat entirely.
- gus_massa 7y ago
  > I have never met a pilot that didn't religiously follow their checklists. Someone forgot to follow the second page of the checklist of a 737 in Argentina in 1999 https://en.wikipedia.org/wiki/LAPA_Flight_3142 https://en.wikipedia.org/wiki/LAPA_Flight_3142
  - jerf 7y ago
    Well, you won't have met them then: "The captain of the flight was Captain Gustavo R. Weigel (45) and the co-pilot was Luis Etcheverry (31). The captain had 6,500 hours of flying experience, of which 1,700 had been in a Boeing 737, the type of aircraft that was in the accident. The co-pilot had flown about 600 of his 4,000 hours in the 737. Both pilots died in the accident."
    - gus_massa 7y ago
      I know. My university is nearby and the bus I used in that years passed by the street the plane crossed before crashing. I took the bus next day and the street was full of debris an the investigation team was still blocking it, so the bus had to take a long detour around the airport. The part about the checklist is more clear in the Spanish version of the article, but it's not 100% clear, now I'm not sure if it was confirmed or it was only a rumor. Something like they forget to read the second part of the checklist because they were distracted by someone, so they forget to open the flags. When the alarm sound, they ignore it because they have a similar alarm that always sound due to a malfunction. There were also some logistic problems, like a schedule with less rest time than required. As always, there is more than only one cause.
stcredzero 7y ago
There's a "trust but verify" element of airplane automation that is ingrained in you as a pilot... What we're talking about here is several seconds heads-up with visual AND instrument confirmation, that the computers are doing what they're supposed to before we allow the plane to land. Let's say we never get to full autonomous driving on roads mixing autonomous cars and human drivers. In that case, is it ultimately going to be that different from the way we teach people to look both ways before they cross the street, and teach new drivers how to deal with highway on-ramps and intersections?
hector_vasquez 7y ago
For many years, manually flown aircraft have required radically different safety practices from manually driven cars. I wonder why you think the addition of automation changes that.

ozim 7y ago

Self driving car approach has to be different because whole idea is different than airplane automation. You want self driving car to be autonomus without need to rely on trained and disciplined human element. End goal is totally different and you cannot make progress to have fully autonomous car if you would come from: "yeah human will be in control all the time, not being stupid or drunk".

nradov 7y ago
It's by no means clear that the end goal and approach should be different. Personally I think that aiming toward level 4-5 autonomous vehicles right now is misguided and unlikely to succeed (outside of a few controlled access roads in limited areas). A better approach would be to require the human driver to actively control the vehicle at all times, but have the computer step in to brake or steer when it detects a high probability of a dangerous situation. Like how auto GCAS works on modern aircraft.
- MR4D 7y ago
  This, completely. A few months ago, I lightly rear-ended a vehicle ahead of me when they stopped at a green light while turning. Legally, I was in the wrong, and it was a stupid mistake. I had looked left to make sure no traffic was coming at the exact moment the person ahead of me stopped. Had either event (my head turning or his breaking) been different by about a second, no accident would have happened. A simple anti-collision system would have prevented this, and saved me some money (and him from some scratches on his bumper). As I have an older car - 2012 model - so this feature wasn't prevalent when I bought it. But my next car will have this capability, if for no other reason than I hate stupid mistakes. This one feature would make my drive to/from work much less stressful (there are a ton of people who will switch lanes in front of you at the last minute and then hit the brakes here in Houston). Add in lane-change warnings, and I think a significant number of accidents could be avoided. They may be small accidents, but a bumper replacement or painting (times 2 - one for each car involved) is easily $1,000, if not more.
  - perl4ever 7y ago
    If you get used to driving a car that actually does this sort of thing well, then the first time you get in a car where it malfunctions or doesn't have the feature, you will have a bad accident because your reflexes will be missing. I got a car that automatically closes the door so you don't have to slam it. Now I keep forgetting to close the door on a normal car properly. It's eliminated my muscle memory in just a few months. Another instance was when I got a car without a clutch interlock. I never would have realized how dependent I was on that until I started it up in gear.
    - MR4D 7y ago
      I agree 100%. To me, I think two possible outcomes will prevail: 1 - All cars add this feature either through regulation or other pressure. 2 - Most cars add this feature, and those that don't see higher accident rates, and therefore higher insurance bills. That should result in those cars being less desirable (and therefore less profitable for the car companies), OR the car companies figure it out and add the missing feature to the rest of their cars. Either way, I think we'll end up there, just not sure of what path we'll go down.

tlb 7y ago

I'm not sure what you're proposing instead. If you built self-driving cars that required pilot-level skills and training to operate them, who would use them?

wil421 7y ago
The main point He is trying to make is the AP has several redundant systems and 2-3 humans double checking the instruments and AP. With self driving cars we use some fancy ML/AI where we don’t exactly know why the machine chose to do what it did.
- cblades 7y ago
  Those aren't comparable things though. Our driver training just flat-out isn't robust enough to support a similar system, and there's only one person in control of the car. He's also leaving out checklist-driven procedure, which is a huge component of airplane safety that is entirely absent from car driving.
  - base698 7y ago
    A scan where you verify what the computer is telling you and cross checking it with what you're seeing could be. Maybe they have a UI showing the path the computer is taking and bounding boxes of all the potential obstacles overlayed on the screen. If there is a disagreement the human takes over. Similar to how if on an instrument approach the altitude doesn't agree with the vertical guidance. You can check other systems to see which instrument has failed and whether it's the vertical guidance or altimeter. Of course, this method of driving and having to pay more attention doesn't jive with the fantasy of getting wasted and having a computer take you home.
    - magduf 7y ago
      >Of course, this method of driving and having to pay more attention doesn't jive with the fantasy of getting wasted and having a computer take you home. Which is why the system you're describing will never come to cars: American drivers would never use it correctly. If it isn't something that complete morons can jump in and use with zero training, it's just not going to work here. What you're describing requires a competent pilot with training, and that's a completely unrealistic expectation for a car in America.

jhayward 7y ago

It's difficult to see what your main point is, but what I take from it is the idea that aircraft autopilots and self-driving cars are two completely separate problem domains, with very different requirements, resources, and operating regimes. It would seem to be foolish to have a goal of having "no fatalities per 100 billion miles", or whatever the current spec level for airline travel is, as the minimum operating requirement for automobiles. Likewise requiring two expert-level operators at the controls, with re-certification every six months seems to be out of scope. So why should the AP try to be designed assuming those things are true?

kazinator 7y ago

Airplanes don't have to react to a three-year-old on a tricycle suddenly emerging from between parked cars.

JamesBarney 7y ago

Autonomous car's don't have to be nearly as safe as airplanes because existing cars are orders of magnitude less safe than airplanes.

jandrese 7y ago
Isn't that one of the problems autonomous cars are trying to solve?
- stcredzero 7y ago
  Would just one order of magnitude improvement count as "solved?"
- kemitche 7y ago
  The point is that cars don't need to make an immediate jump from "current levels of safety" to "plane levels of safety." Anything that moves the needle noticeably closer to "plane levels" is a solid improvement in terms of reducing unnecessary deaths.
  - perl4ever 7y ago
    Why do you think that autonomous cars will start by replacing the worst drivers, rather than replacing professional drivers, who are much better than the average person? Isn't the biggest potential initial demand for autonomous cars people who don't own cars and currently must choose between bus and taxi?
LoSboccacc 7y ago
citation needed. notice how incident per mile statistic where quickly submerged by Tesla around their fourth autopilot death. also note how autopilot are currently d enabled in the less risky part of the journey. also notice how autopilot stat in most articles are compared toward national averages, which include older car with less safety features and bikes.
- 7y ago

nostrademons 7y ago

It's different because the market for (and culture of) pilots is dramatically different from the market for drivers. I have never met a pilot that didn't religiously follow their checklists. If there is ever any doubt as to whether something is safe, there is no doubt: it is not. Weight limits, weather reports, etc. all are carefully attended to. Meanwhile, 30% of adults sitting in the back seat of a car forget to buckle their seatbelts. About 5% admitted to driving drunk within the last month. I see people on their cell phones all their time. I've ridden with a young woman who was driving with her knees while doing her hair in the rearview mirror (yes, I was freaked out, and no, I never rode with her again). Pilots are professionals, and they act professionally, and they respect the possibility of death every time they get into their vehicle. Drivers are not. There's a long history of every safety innovation in cars actually making accident rates go up, because drivers start taking them for granted and then behaving in ways that compensate for the increased safety and maintain the same level of subjective risk (like driving faster, driving more, or becoming more distracted). The only way to break that loop is to take the human out of driver's seat entirely.

gus_massa 7y ago
> I have never met a pilot that didn't religiously follow their checklists. Someone forgot to follow the second page of the checklist of a 737 in Argentina in 1999 https://en.wikipedia.org/wiki/LAPA_Flight_3142 https://en.wikipedia.org/wiki/LAPA_Flight_3142
- jerf 7y ago
  Well, you won't have met them then: "The captain of the flight was Captain Gustavo R. Weigel (45) and the co-pilot was Luis Etcheverry (31). The captain had 6,500 hours of flying experience, of which 1,700 had been in a Boeing 737, the type of aircraft that was in the accident. The co-pilot had flown about 600 of his 4,000 hours in the 737. Both pilots died in the accident."
  - gus_massa 7y ago
    I know. My university is nearby and the bus I used in that years passed by the street the plane crossed before crashing. I took the bus next day and the street was full of debris an the investigation team was still blocking it, so the bus had to take a long detour around the airport. The part about the checklist is more clear in the Spanish version of the article, but it's not 100% clear, now I'm not sure if it was confirmed or it was only a rumor. Something like they forget to read the second part of the checklist because they were distracted by someone, so they forget to open the flags. When the alarm sound, they ignore it because they have a similar alarm that always sound due to a malfunction. There were also some logistic problems, like a schedule with less rest time than required. As always, there is more than only one cause.

stcredzero 7y ago

There's a "trust but verify" element of airplane automation that is ingrained in you as a pilot... What we're talking about here is several seconds heads-up with visual AND instrument confirmation, that the computers are doing what they're supposed to before we allow the plane to land. Let's say we never get to full autonomous driving on roads mixing autonomous cars and human drivers. In that case, is it ultimately going to be that different from the way we teach people to look both ways before they cross the street, and teach new drivers how to deal with highway on-ramps and intersections?

hector_vasquez 7y ago

For many years, manually flown aircraft have required radically different safety practices from manually driven cars. I wonder why you think the addition of automation changes that.