4 ms·
There's a weird vibe in this article I don't like. It (correctly) notes that as the number of anomalies that the automation passes on to the human operators goe
by munificent 10y ago
There's a weird vibe in this article I don't like. It (correctly) notes that as the number of anomalies that the automation passes on to the human operators goes down, the rate that humans successfully handle them also goes down.
But it doesn't seem to do a good job of clarifying that the total number of incorrectly handled anomalies is still decreasing. Let's say your automation goes from handling 90% to 99% of the anomalies and that when it does handle one, it does so correctly. We'll say that the increased rarity of human interaction and the inattention and weakened training that causes makes the human pilots to go from being able to handle 90% of them correctly to only a terrifying 40%.
Let's run the simulation. With the old automation:
1000 anomalies occur
100 (10%) make it past the automation
10 (10%) make it past the human operators
So 10 catastrophes. Now with the new moderately better automation and much worse human performance:
1000 anomalies occur
10 (1%) make it past the automation
6 (60%) make it past the human operators
6 catastrophes. Even though the human performance was much worse, because they are the last stage in the pipeline, it has a lower effect.
Now, I just pulled these numbers out of my ass, but I think it's important to focus on the total number of automation+human failures and not single out one stage or the other. From the passenger's perspective, they don't care who saved their ass, just that it got saved. If we can make one stage more failure proof at the expense of the other, it can still be a net win.
- galdosdi 10y agoAlso, there are things organizations can do to combat the problem of human operators getting rusty. Practice. A lot of organizations just don't do it though because it's too tempting to view the automation's cost savings as "free" and just take them for granted, but it can help a lot.
- thaumasiotes 10y agoPeople get rusty for a reason. Your solution suffers from a couple of problems: - Practice isn't the same thing as actual events. Being good at practice is more likely to diverge from being good at crisis response as actual crises become rare, because the criterion of matching what would happen in a crisis gets much less important. Thus, peacetime militaries often need radical overhauling before they can really get much accomplished when war breaks out. (Also consider - we have a lot of people who are really interested in how medieval combat (whether in a battle line or a duel) worked, what effective use of weapons looked like, and so on. But for all the discussion, we don't know, and we can't know unless we actually stage regular battles-to-the-death with period tooling. One form of combat practice, however, has been preserved as European fencing. How closely does it correspond? Again, we don't know, but consensus is "not well".) - The automation's cost savings are free -- in fact, in this example, they have a large negative cost, cutting catastrophes by 40%. Keeping everyone in shape to handle crises they're likely to never actually see is, arguably, an enormous waste of money. (In addition to actually being impossible much of the time, as in my first bullet point.)
- vorotato 10y agoon empty legs (flights back with no passengers) you could disable automation as protocol.
- r00fus 10y agoA very insightful idea. You could also measure this by ensuring pilots have a certain number of hours logged without automation (practice by pudding). Are there analogies in other industries to this approach?
- ChoHag 10y agoNever pulled the plug on your UPS to make sure you can or ripped out half the raid array without warning?
- JoshTriplett 10y ago> Are there analogies in other industries to this approach? https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey
- galdosdi 10y ago- Yeah, practice is definitely not the same, but depending on the domain, it's usually at least a lot better than nothing, even if the real thing is a lot better than practice. (To speak to your example, at least in wartime, peacetime militaries don't have to worry about reteaching their soldiers EVERYTHING. Maybe they've never been in real combat, but at least they can consistently shoot at a target. That's better than not being able to do that either.) - I mean, it depends on the specific case, right? Obviously that's true in this example. You can also come up with opposing examples. I also misspoke a bit -- even if the automation cost savings are free and you're strictly better off with it than without it, adding some practice for human operators may get you even more savings, and it's often overlooked. My point is just that practice/drilling can be a useful tool in the toolbox. It depends on the situation, but it shouldn't be ignored.
- yummyfajitas 10y agoEven though the human performance was much worse, because they are the last stage in the pipeline, it has a lower effect. Minor mathematical nit: it's not because of which stage of the pipeline you are in. Multiplication is commutative.