5 ms·

I wonder if increased automation means poorer responses to emergencies because you have fewer humans there to respond with. On the other hand, maybe it means fe

I wonder if increased automation means poorer responses to emergencies because you have fewer humans there to respond with. On the other hand, maybe it means fewer emergencies because the automation makes fewer mistakes, or makes hard jobs easier and more consistent.

kbenson 10y ago
Often overlooked is the type of mistake, which it seems you are touching on with "poorer responses to emergencies". If humans make 10 mistakes over a time period compared to 1 for the automated system, it might look as if the humans are worse, but if the cost of the human mistakes averages to 5 for some theoretical cost scale (so 50 overall), but the automated mistake costs 100 because there isn't a good dynamic response to it, the automated system doesn't actually cost less in the end. That's why we so often see automated systems backed by people, as we don't have automation that can respond as dynamically as a person. It usually takes far fewer people to oversee automation than to do the original work though.
- porpoisemonkey 10y ago
  > That's why we so often see automated systems backed by people, as we don't have automation that can respond as dynamically as a person. I imagine there's specialized training involved to make sure that the person monitoring the machine understands the nature of how the machine operates in order to service it and know when it's making mistakes. I wonder if the people monitoring the machines are the same ones that are being automated out of jobs or if it's an entirely different skill set?
- bluGill 10y ago
  It only takes one human mistake that is 1000 to destroy all that. Many systems in oil refineries are automated because humans didn't want to shut the plant down where there was some trouble. Even when the problem is a predictable humans often refused to admit it they couldn't operate as is. (We know this pump is failing, the replacement won't arrive for 2 months though so we are going to continue as long as we can - once it fails we know we have 1 hour to hit the shutdown switch or the plant will explode)
  - kbenson 10y ago
    > It only takes one human mistake that is 1000 to destroy all that. And it only takes one machine mistake that's 10000 to reverse it again. What it comes down to is that until every mistake that can happen is known and accounted for, a human somewhere in the process will allow for better reasoning about any possible mistakes. Human errors are more likely, but they are fairly well understood and you can safeguard yourself by putting more people higher up the stack checking things. This is well understood, we've had centuries (if not millennia, depending on how you want to qualify it) to work on the problem of reducing errors in people's work. > Many systems in oil refineries are automated because humans didn't want to shut the plant down where there was some trouble. That's not a problem with people, that's a problem with the structure the people are in. When keeping the operation running is prioritized over fixing problems as they are found, you get a perverse incentive structure. A good example of this is shown in the NUMMI episode[1] of This American Life. In the 1980's at the NUMMI plant, GM never wanted the assembly line stopped, so cars would go through not completely finished, and then workers would volunteer to finish them on the lot manually (for extra money), but theyse cars often had problems and were very low quality. This is contrasted with what the GM execs saw at a Toyota plant (and tries to institute), which was that any problems on the line were encouraged to stop or slow the line and managers would then go investigate and either fix problems with that station, or allocate more resources there if it wasn't sufficient. The problem you outline is not inherent to the people working, the problem is the structure they are working within. The main difference in reality is that when an automated component isn't working right, people know it (or something associated with it, such as a feeder) needs to be fixed, so they fix it. Or more appropriately, they've been trained that this is the correct course of action. The same could be true of people working, it just often wasn't. 1: https://www.thisamericanlife.org/radio-archives/episode/403/nummi https://www.thisamericanlife.org/radio-archives/episode/403/...
    - stickfigure 10y ago
      Human errors are more likely, but they are fairly well understood and you can safeguard yourself by putting more people higher up the stack checking things. This doesn't sound true to me. The worst man-made catastrophes seem to frequently involve a human manually disabling some sort of critical safety mechanism (eg, Chernobyl). You could look at the checklist as a primitive form of automation that helps precisely because it removes human judgement. In theory, automation could encompass the knowledge of the finest experts in the field. Humans get credit for being able to think flexibly and accommodate information that maybe wasn't included in the original programming. But if you're experiencing a six-sigma event that the system wasn't programmed for, what is the chance that Joe Random Operator (completed a two-month training certificate program) is going to be savvy enough to respond better?

kbenson 10y ago

Often overlooked is the type of mistake, which it seems you are touching on with "poorer responses to emergencies". If humans make 10 mistakes over a time period compared to 1 for the automated system, it might look as if the humans are worse, but if the cost of the human mistakes averages to 5 for some theoretical cost scale (so 50 overall), but the automated mistake costs 100 because there isn't a good dynamic response to it, the automated system doesn't actually cost less in the end. That's why we so often see automated systems backed by people, as we don't have automation that can respond as dynamically as a person. It usually takes far fewer people to oversee automation than to do the original work though.

porpoisemonkey 10y ago
> That's why we so often see automated systems backed by people, as we don't have automation that can respond as dynamically as a person. I imagine there's specialized training involved to make sure that the person monitoring the machine understands the nature of how the machine operates in order to service it and know when it's making mistakes. I wonder if the people monitoring the machines are the same ones that are being automated out of jobs or if it's an entirely different skill set?
bluGill 10y ago
It only takes one human mistake that is 1000 to destroy all that. Many systems in oil refineries are automated because humans didn't want to shut the plant down where there was some trouble. Even when the problem is a predictable humans often refused to admit it they couldn't operate as is. (We know this pump is failing, the replacement won't arrive for 2 months though so we are going to continue as long as we can - once it fails we know we have 1 hour to hit the shutdown switch or the plant will explode)
- kbenson 10y ago
  > It only takes one human mistake that is 1000 to destroy all that. And it only takes one machine mistake that's 10000 to reverse it again. What it comes down to is that until every mistake that can happen is known and accounted for, a human somewhere in the process will allow for better reasoning about any possible mistakes. Human errors are more likely, but they are fairly well understood and you can safeguard yourself by putting more people higher up the stack checking things. This is well understood, we've had centuries (if not millennia, depending on how you want to qualify it) to work on the problem of reducing errors in people's work. > Many systems in oil refineries are automated because humans didn't want to shut the plant down where there was some trouble. That's not a problem with people, that's a problem with the structure the people are in. When keeping the operation running is prioritized over fixing problems as they are found, you get a perverse incentive structure. A good example of this is shown in the NUMMI episode[1] of This American Life. In the 1980's at the NUMMI plant, GM never wanted the assembly line stopped, so cars would go through not completely finished, and then workers would volunteer to finish them on the lot manually (for extra money), but theyse cars often had problems and were very low quality. This is contrasted with what the GM execs saw at a Toyota plant (and tries to institute), which was that any problems on the line were encouraged to stop or slow the line and managers would then go investigate and either fix problems with that station, or allocate more resources there if it wasn't sufficient. The problem you outline is not inherent to the people working, the problem is the structure they are working within. The main difference in reality is that when an automated component isn't working right, people know it (or something associated with it, such as a feeder) needs to be fixed, so they fix it. Or more appropriately, they've been trained that this is the correct course of action. The same could be true of people working, it just often wasn't. 1: https://www.thisamericanlife.org/radio-archives/episode/403/nummi https://www.thisamericanlife.org/radio-archives/episode/403/...
  - stickfigure 10y ago
    Human errors are more likely, but they are fairly well understood and you can safeguard yourself by putting more people higher up the stack checking things. This doesn't sound true to me. The worst man-made catastrophes seem to frequently involve a human manually disabling some sort of critical safety mechanism (eg, Chernobyl). You could look at the checklist as a primitive form of automation that helps precisely because it removes human judgement. In theory, automation could encompass the knowledge of the finest experts in the field. Humans get credit for being able to think flexibly and accommodate information that maybe wasn't included in the original programming. But if you're experiencing a six-sigma event that the system wasn't programmed for, what is the chance that Joe Random Operator (completed a two-month training certificate program) is going to be savvy enough to respond better?