3 ms·

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's h

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and will end very badly. The person whose Tesla drove into a concrete barrier at 65 mph earlier this year was perfectly capable of driving a car, but they mistakenly believed that the computer had it under control. 99% Invisible has a pair of podcast episodes on the subject (2015): https://99percentinvisible.org/episode/children-of-the-magenta-automation-paradox-pt-1/ https://99percentinvisible.org/episode/children-of-the-magen... https://99percentinvisible.org/episode/johnnycab-automation-paradox-pt-2/ https://99percentinvisible.org/episode/johnnycab-automation-...

ballenf 8y ago
Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls. Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated peer review.
- FussyZeus 8y ago
  I would argue against this one, simply because of the Airbus incident from some time ago where to avoid pushing feedback into the controls, the plane was designed to average out the inputs from the two pilots. The older, more experienced pilot was attempting to maneuver the plane in such a way to correct their problem (I believe it was a stall) and the younger pilot was panicking and attempting the opposite maneuver, the plane averaged it to roughly no course adjustment, and they hit a mountainside. I'm all for automating driving and other such tasks once the computers are ready, but until we know they're at least most likely ready to do it, I want the ability to turn it off and do it myself. (And frankly, I want that ability anyway because I genuinely love driving and it makes me sad to think someday I won't be able to do it.)
  - wlesieutre 8y ago
    For reference, I believe this is the crash you're referring to: https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447 The fly-by-wire control system ordinarily prevents stalls, but it had disengaged due to an iced over sensor and was operating without stall protection. The plane stalled at 38,000 feet and fell into the ocean. They should have had plenty of time to correct the stall, but one pilot was pulling back his control stick (the opposite of what they needed to do), and since the two sticks aren't physically linked the other pilot didn't know he was doing that. It's one of the topics discussed in the podcast that I linked above.
    - FussyZeus 8y ago
      Yeah that's the one.
- wlesieutre 8y ago
  Even if you let the computer run the show wholesale it could work out net positive. Autopilot might cause problems, but it might also prevent twice as many accidents that would have been caused by pilot error. We're still in pretty early days here, so I don't know if Watson's advice is at a point where that could be true. In the meantime, I do like the idea of having an unaided doctor and the computer program evaluate it independently, then afterward say "Ok, let's see what the computer thought" before coming to any final conclusions. Maybe it'd help avoid any dangerous recommendations. Or maybe people would say "Well the computer has a lot more data than me, it's probably right." I think that depends on whether the differences are errors similar to what a human would make, or if they're the "oh my god how did it even come up with this result" nonsensical error that are easily spotted by human review.
raincom 8y ago
That's one of the implicit reasons why US based airlines in the past used to recruit retired pilots from U.S Air Force. In case instruments fail or contradictory signals are present, pilots with physical skills (kinesthesia, etc) can steer away from dangers. Two incidents demonstrate that: 1. https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214 https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214 2. https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447 Pilots on both came from the civilian background. The difference between civilian training and combat training is that the latter is more trained to operate in conditions that involve instruments failure, distress, etc.
- ApolloFortyNine 8y ago
  Aren't both of those just examples of pilot error, regardless of autopilot? In the case of the Asiana flight, the captain chose a visual approach when he could have let the autopilot land it. And from there it's simple pilot error. In the case of the air France flight, one of the pilots chose to ignore a stall warning and pull up instead of pushing down to gain speed. Sadly, in the case of no pilot input the plane actually would have ended up avoiding the stall on its own. I of course agree that military pilots with real experience will likely perform better than civilian pilots, I just doubt autopilot had any part to play in either of those accidents, like the comment you replied to implies.
- sjm-lbm 8y ago
  There's also been a lot of discussion and research about how to best interact with autopilot, and when the best times to turn it off and operate the airplane manually are. There is a talk[1] about this that is really popular among pilots, and I honestly think it seems increasingly relevant to the rest of the world as more and more things become computer automated in some fashion. [1] - https://www.youtube.com/watch?v=pN41LvuSz10 https://www.youtube.com/watch?v=pN41LvuSz10
deleted 8y ago
[deleted]
mrnobody_67 8y ago
That's the difference between Oracles and Agents. Oracle = make suggestion Agent = Act on your behalf. Waze might be starting off as an Oracle. but once you tie it into self driving cars, it'll become an agent acting on your behalf. At a certain tipping point, creates really interesting questions. For example, if you re-direct 100% of traffic to less congested route, you end up creating traffic jams, so then you have to decide who goes on which route... maybe people who pay more get faster routing, or people deemed more important or busier by the algorithm...

ballenf 8y ago

Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls. Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated peer review.

FussyZeus 8y ago
I would argue against this one, simply because of the Airbus incident from some time ago where to avoid pushing feedback into the controls, the plane was designed to average out the inputs from the two pilots. The older, more experienced pilot was attempting to maneuver the plane in such a way to correct their problem (I believe it was a stall) and the younger pilot was panicking and attempting the opposite maneuver, the plane averaged it to roughly no course adjustment, and they hit a mountainside. I'm all for automating driving and other such tasks once the computers are ready, but until we know they're at least most likely ready to do it, I want the ability to turn it off and do it myself. (And frankly, I want that ability anyway because I genuinely love driving and it makes me sad to think someday I won't be able to do it.)
- wlesieutre 8y ago
  For reference, I believe this is the crash you're referring to: https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447 The fly-by-wire control system ordinarily prevents stalls, but it had disengaged due to an iced over sensor and was operating without stall protection. The plane stalled at 38,000 feet and fell into the ocean. They should have had plenty of time to correct the stall, but one pilot was pulling back his control stick (the opposite of what they needed to do), and since the two sticks aren't physically linked the other pilot didn't know he was doing that. It's one of the topics discussed in the podcast that I linked above.
  - FussyZeus 8y ago
    Yeah that's the one.
wlesieutre 8y ago
Even if you let the computer run the show wholesale it could work out net positive. Autopilot might cause problems, but it might also prevent twice as many accidents that would have been caused by pilot error. We're still in pretty early days here, so I don't know if Watson's advice is at a point where that could be true. In the meantime, I do like the idea of having an unaided doctor and the computer program evaluate it independently, then afterward say "Ok, let's see what the computer thought" before coming to any final conclusions. Maybe it'd help avoid any dangerous recommendations. Or maybe people would say "Well the computer has a lot more data than me, it's probably right." I think that depends on whether the differences are errors similar to what a human would make, or if they're the "oh my god how did it even come up with this result" nonsensical error that are easily spotted by human review.

raincom 8y ago

That's one of the implicit reasons why US based airlines in the past used to recruit retired pilots from U.S Air Force. In case instruments fail or contradictory signals are present, pilots with physical skills (kinesthesia, etc) can steer away from dangers. Two incidents demonstrate that: 1. https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214 https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214 2. https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447 Pilots on both came from the civilian background. The difference between civilian training and combat training is that the latter is more trained to operate in conditions that involve instruments failure, distress, etc.

ApolloFortyNine 8y ago
Aren't both of those just examples of pilot error, regardless of autopilot? In the case of the Asiana flight, the captain chose a visual approach when he could have let the autopilot land it. And from there it's simple pilot error. In the case of the air France flight, one of the pilots chose to ignore a stall warning and pull up instead of pushing down to gain speed. Sadly, in the case of no pilot input the plane actually would have ended up avoiding the stall on its own. I of course agree that military pilots with real experience will likely perform better than civilian pilots, I just doubt autopilot had any part to play in either of those accidents, like the comment you replied to implies.
sjm-lbm 8y ago
There's also been a lot of discussion and research about how to best interact with autopilot, and when the best times to turn it off and operate the airplane manually are. There is a talk[1] about this that is really popular among pilots, and I honestly think it seems increasingly relevant to the rest of the world as more and more things become computer automated in some fashion. [1] - https://www.youtube.com/watch?v=pN41LvuSz10 https://www.youtube.com/watch?v=pN41LvuSz10

deleted 8y ago

[deleted]

mrnobody_67 8y ago

That's the difference between Oracles and Agents. Oracle = make suggestion Agent = Act on your behalf. Waze might be starting off as an Oracle. but once you tie it into self driving cars, it'll become an agent acting on your behalf. At a certain tipping point, creates really interesting questions. For example, if you re-direct 100% of traffic to less congested route, you end up creating traffic jams, so then you have to decide who goes on which route... maybe people who pay more get faster routing, or people deemed more important or busier by the algorithm...