6 ms·

There's a recent econtalk episode that delves into transferring control from machine to human. Specifically, regarding the air france disaster: "The Air Franc

by drhodes 10y ago

There's a recent econtalk episode that delves into transferring control from machine to human. Specifically, regarding the air france disaster: "The Air France story is a story about a failed handoff, where the automation onboard an airplane found a relatively minor fault and handed control of the plane back to the human pilots, too suddenly and ungracefully surprised them. And they had lost some of their skills flying too much with the automated systems and lost control of the airplane. Which actually was a perfectly good airplane about a minute into the crisis. And so they went from tens of thousands of feet flying through the sky and ended up spiraling into the ocean, tragically losing all aboard." http://www.econtalk.org/archives/2015/11/david_mindell_o.html http://www.econtalk.org/archives/2015/11/david_mindell_o.htm... https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447

jfoutz 10y ago
That tragedy was probably the seed of my concern. Lately i've been thinking a lot about how automated systems become brittle. The systems don't change, people forget the dependencies and requirements and in something real time like an airplane, the feel of the system. A configuration management system that incorporated spaced repetition would be cool. Every once in a while, go into an incremental mode where you actually type in the commands. It has the added bonus of getting new people aware of the system. Sure, you can always just go read the source and figure it out. Having the system force some human awareness from time to time would be handy.
- fencepost 10y ago
  Perhaps all critical automated systems should have some level of Chaos Monkey testing built into them, though it might be best if it was good enough to not fail even if that test was failed.
  - jfoutz 10y ago
    Chaos Monkey is great. I'm suggesting something slightly different. Imagine you have a bunch of cent 6 machines that have been running great for years and years. upgrades go smoothly and deployment is a breeze. Now you need to upgrade to cent 7. Stuff that used to happen in init.d now happens in the systemd, for example. Reviewing changes by hand every few months keeps you aware of what needs to happen. Automation doesn't get brittle, people forget how systems are flexible and inflexible. Spaced repetition would give people a chance to keep up with the current design, so when that crazy security vulnerability happens, you can jump in and change stuff knowing how it works, rather than having to figure it out on the fly. I don't think it would take much. Configure a box today, tomorrow, next week, next month, 3 months, then perhaps every 6 months. There are very smart people that sort of intuitively keep up with those kinds of changes. But those people change jobs. How do you get a mere mortal up to speed on 20-30 machine configurations? config management can configure hundreds of machines a day, no problem. but it's easy to forget what's really going into each of those boxes. This is more about preserving organizational awareness, not so much robustness of a running system.
- ethbro 10y ago
  Why hand control back to a human when you can do even better? Have them both handling control at the same time, reconcile the inputs in a sane manner (or have a master/slave where one is providing phantom inputs). Added benefit of being able to error check each other. https://en.m.wikipedia.org/wiki/Lockstep_(computing) https://en.m.wikipedia.org/wiki/Lockstep_(computing)
  - koenigdavidmj 10y ago
    This was exactly the problem in the Airbus crash shown above; reconciling the inputs is hard. One pilot kept holding one stick back, unbeknownst to the other. This plane averages them by default, and therefore the plane stalled. That couldn't happen on a Boeing because the yokes are yokes (inputs on one are easily visible to both pilots) and physically linked. (Supposedly the plane is supposed to loudly complain if the inputs diverge. I'm not sure if this happened.)
    - theoh 10y ago
      The Airbus did make the expected warnings ("DUAL INPUT") when the control inputs conflicted, at least according to this Vanity Fair article: http://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash http://www.vanityfair.com/news/business/2014/10/air-france-f...
      - saint_fiasco 10y ago
        "DUAL INPUT" is only marginally better than "PC LOAD LETTER"
        schoen 10y ago
        On the other hand, typical Airbus A330 operators probably have a lot better training about what diagnostic messages mean than typical HP LaserJet operators do.
        ceejayoz 10y ago
mrpippy 10y ago
The Langewiesche article is a great read as well, with more details: http://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash http://www.vanityfair.com/news/business/2014/10/air-france-f...
- cmurf 10y ago
  It's shit. https://www.psychologytoday.com/blog/conquer-fear-flying/201411/pilot-and-blogger-patrick-smith-sets-the-record-straight https://www.psychologytoday.com/blog/conquer-fear-flying/201...
spuz 10y ago
Thanks, that's a really interesting episode.
imperialdrive 10y ago
http://archive.ncsa.illinois.edu/prajlich/forster.html http://archive.ncsa.illinois.edu/prajlich/forster.html THE MACHINE STOPS

by drhodes 10y ago

jfoutz 10y ago

That tragedy was probably the seed of my concern. Lately i've been thinking a lot about how automated systems become brittle. The systems don't change, people forget the dependencies and requirements and in something real time like an airplane, the feel of the system. A configuration management system that incorporated spaced repetition would be cool. Every once in a while, go into an incremental mode where you actually type in the commands. It has the added bonus of getting new people aware of the system. Sure, you can always just go read the source and figure it out. Having the system force some human awareness from time to time would be handy.

fencepost 10y ago
Perhaps all critical automated systems should have some level of Chaos Monkey testing built into them, though it might be best if it was good enough to not fail even if that test was failed.
- jfoutz 10y ago
  Chaos Monkey is great. I'm suggesting something slightly different. Imagine you have a bunch of cent 6 machines that have been running great for years and years. upgrades go smoothly and deployment is a breeze. Now you need to upgrade to cent 7. Stuff that used to happen in init.d now happens in the systemd, for example. Reviewing changes by hand every few months keeps you aware of what needs to happen. Automation doesn't get brittle, people forget how systems are flexible and inflexible. Spaced repetition would give people a chance to keep up with the current design, so when that crazy security vulnerability happens, you can jump in and change stuff knowing how it works, rather than having to figure it out on the fly. I don't think it would take much. Configure a box today, tomorrow, next week, next month, 3 months, then perhaps every 6 months. There are very smart people that sort of intuitively keep up with those kinds of changes. But those people change jobs. How do you get a mere mortal up to speed on 20-30 machine configurations? config management can configure hundreds of machines a day, no problem. but it's easy to forget what's really going into each of those boxes. This is more about preserving organizational awareness, not so much robustness of a running system.
ethbro 10y ago
Why hand control back to a human when you can do even better? Have them both handling control at the same time, reconcile the inputs in a sane manner (or have a master/slave where one is providing phantom inputs). Added benefit of being able to error check each other. https://en.m.wikipedia.org/wiki/Lockstep_(computing) https://en.m.wikipedia.org/wiki/Lockstep_(computing)
- koenigdavidmj 10y ago
  This was exactly the problem in the Airbus crash shown above; reconciling the inputs is hard. One pilot kept holding one stick back, unbeknownst to the other. This plane averages them by default, and therefore the plane stalled. That couldn't happen on a Boeing because the yokes are yokes (inputs on one are easily visible to both pilots) and physically linked. (Supposedly the plane is supposed to loudly complain if the inputs diverge. I'm not sure if this happened.)
  - theoh 10y ago
    The Airbus did make the expected warnings ("DUAL INPUT") when the control inputs conflicted, at least according to this Vanity Fair article: http://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash http://www.vanityfair.com/news/business/2014/10/air-france-f...
    - saint_fiasco 10y ago
      "DUAL INPUT" is only marginally better than "PC LOAD LETTER"
      - schoen 10y ago
        On the other hand, typical Airbus A330 operators probably have a lot better training about what diagnostic messages mean than typical HP LaserJet operators do.
        ceejayoz 10y ago

mrpippy 10y ago

The Langewiesche article is a great read as well, with more details: http://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash http://www.vanityfair.com/news/business/2014/10/air-france-f...

cmurf 10y ago
It's shit. https://www.psychologytoday.com/blog/conquer-fear-flying/201411/pilot-and-blogger-patrick-smith-sets-the-record-straight https://www.psychologytoday.com/blog/conquer-fear-flying/201...

spuz 10y ago

Thanks, that's a really interesting episode.

imperialdrive 10y ago

http://archive.ncsa.illinois.edu/prajlich/forster.html http://archive.ncsa.illinois.edu/prajlich/forster.html THE MACHINE STOPS