4 ms·

Automation is incapable of "judgement;" automation consistently and reliably responds to inputs according to pre-defined instructions. That's why automation wo

Automation is incapable of "judgement;" automation consistently and reliably responds to inputs according to pre-defined instructions. That's why automation works really well in some situations but not others. The more complex the task, the harder it is make a complete instruction set that will result in a satisfactory outcome for every possible situation. Even human pilots behave somewhat like automatons in some situations: in most situations they follow procedures, which could be described as "responding to inputs according to pre-defined instructions." However, they often encounter situations not covered by the procedures, in which case they must instead exercise their judgement. The problem with judgement is that it is neither consistent nor reliable. Some humans have better judgement than others. Eventually we will have automation sophisticated enough to handle even to full complexity of aviation, at which point automation will yield safe results more consistently and reliably than human judgement.

weavejester 15y ago
> Automation is incapable of "judgement;" automation consistently and reliably responds to inputs according to pre-defined instructions. Where are you getting this definition of "automation" from? As far as I'm concerned, an automatic system is simply one that require little or no direct human control.
_delirium 15y ago
I suppose it gets into philosophical debates about AI, but I don't see any reason in principle that we can't describe at least some kinds of computerized systems as exercising something described as "judgment". We already have one existence proof, the human brain, of a system that can exercise something we call "judgment", and I don't see a strong reason to believe that it's due to anything magical about the human brain in particular (like a soul or something along those lines), rather than just being a complex reasoning system that's able to balance many contextual factors.
- lucasjung 15y ago
  Automated systems as they currently exist have fixed responses to fixed inputs. If an automatic system encounters a set of inputs it wasn't programmed for, it has no capacity to determine a best course of action for those inputs. Depending on how it was programmed, it will either keep doing what it was doing previously, or switch to a pre-programmed "contingency plan" that hopefully will result in a tolerable outcome (but which might result in catastrophe), or possibly execute a random set of instructions (which can happen to poorly-designed state machines, for example). A human being, on the other hand, has the capacity, when faced with unexpected or unfamiliar conditions, to exercise something we call "judgement" in an attempt to develop an appropriate response. I'm not saying that it's impossible for an automated system to have this capacity, I'm saying that no current automated systems have it, and that we're nowhere near to developing such a system any time soon.
  - _delirium 15y ago
    That's definitely the case with deployed civilian aircraft systems, but I'd be surprised if there wasn't some unmanned system somewhere doing more complex reasoning. There was a talk years ago at IJCAI from some people from NASA Ames on a prototype aircraft-control system they'd built that used a reasoning system to assist with performing emergency landings in situations with no preprogrammed contingency, by taking into account some telemetry information (e.g. aircraft damage), map information, an aerodynamic model, and risk models. I do believe they were planning to deploy it as a suggestion system though, which would suggest a course of action, and then leave it to the pilot to implement it or not. Then the judgment gets more murky; now the system is doing some of the judgment (evaluation of alternatives, etc.) that a human pilot would normally do, but leaving some of the judgment (accept the suggestion, modify it) still to the human. edit: Here's a more recent paper than the one I'm thinking of, but must be the same project: http://ti.arc.nasa.gov/m/profile/de2smith/publications/IAAI09-ELP.pdf http://ti.arc.nasa.gov/m/profile/de2smith/publications/IAAI0...
onemoreact 15y ago
That's not really true. It's often the case that automation is designed reach a specific goal and it then try's to achieve / maintain that state. AKA Segways try to balance and an F-15 Control Augmentation System (CAS) aka stability assistance system try's to keep flying even without a wing. (Yes, this worked and was not programmed for.) http://www.airliners.net/aviation-forums/tech_ops/print.main?id=84282 http://www.airliners.net/aviation-forums/tech_ops/print.main... Plenty of other planes of lost a section of wing a wing and still landed. http://www.airliners.net/aviation-forums/general_aviation/read.main/1252625/ http://www.airliners.net/aviation-forums/general_aviation/re... Granted, all of these cases had a pilot, but in the F-15 the avionics actually discovered how to maintain level flight after the loss of the wing.
- lucasjung 15y ago
  I have a master's in aeronautical engineering, and one of my major areas of study was stability and control, which is what your Segway and F-15 examples fall under. Automated flight control systems definitely do not exercise "judgement:" they have inputs, a transfer function (typically MIMO, these days), and outputs. It used to be that the transfer function was fixed, but more sophisticated systems (e.g. fighter jets) often have many different transfer functions and switch between them based upon various inputs. They don't "decide" or "discover" anything: for any given set of inputs, they will predictably produce a pre-determined set of outputs. Aircraft stability and control systems are not programmed to care about, or even know about, the existance of the wings. The closest they get to this is that they will know the current states of the control surfaces on the wings. So it doesn't really make sense to say that the F-15 CAS was "not programmed for" the state of missing a wing (although it almost certainly was programmed to respond properly in the situation where it gets no feedback from some of the flight controls). As you say, it's designed to reach a specific goal (keep the plane level) and then maintain that state. If the system detects an uncommanded roll-rate, it will move the flight controls to stop that roll-rate. It doesn't know or care that the uncommanded roll-rate is the result of asymmetric lift due to an (almost completely) missing wing: it's just going to keep moving the flight control surfaces until that roll rate goes away. If the aicraft had been damaged in a slightly different way, it's possible that the CAS would have issued commands to the flight controls that would have departed the plane, but fortunately the handling characteristics of the aircraft remained close enough to normal that the control laws still produced good results. Unfortunately, this behavior can result in mishaps when a flight control system gets erroneous inputs: when it believes that it is rolling when it is wings-level (or believies it is level when it is rolling). This was a major contributor to the Air France fligh 447 crash. In such situations, it takes judgement to realize that something is wrong and to figure out what to do about it.
rbanffy 15y ago
Before claiming a machine is incapable of judgment, we would have to define what it is. Is a fly capable of judgment? A fish? This is a philosophical question and won't yield any meaningful answers. Call it judgment of feedback loop, the end result is similar.

weavejester 15y ago

> Automation is incapable of "judgement;" automation consistently and reliably responds to inputs according to pre-defined instructions. Where are you getting this definition of "automation" from? As far as I'm concerned, an automatic system is simply one that require little or no direct human control.

_delirium 15y ago

I suppose it gets into philosophical debates about AI, but I don't see any reason in principle that we can't describe at least some kinds of computerized systems as exercising something described as "judgment". We already have one existence proof, the human brain, of a system that can exercise something we call "judgment", and I don't see a strong reason to believe that it's due to anything magical about the human brain in particular (like a soul or something along those lines), rather than just being a complex reasoning system that's able to balance many contextual factors.

lucasjung 15y ago
Automated systems as they currently exist have fixed responses to fixed inputs. If an automatic system encounters a set of inputs it wasn't programmed for, it has no capacity to determine a best course of action for those inputs. Depending on how it was programmed, it will either keep doing what it was doing previously, or switch to a pre-programmed "contingency plan" that hopefully will result in a tolerable outcome (but which might result in catastrophe), or possibly execute a random set of instructions (which can happen to poorly-designed state machines, for example). A human being, on the other hand, has the capacity, when faced with unexpected or unfamiliar conditions, to exercise something we call "judgement" in an attempt to develop an appropriate response. I'm not saying that it's impossible for an automated system to have this capacity, I'm saying that no current automated systems have it, and that we're nowhere near to developing such a system any time soon.
- _delirium 15y ago
  That's definitely the case with deployed civilian aircraft systems, but I'd be surprised if there wasn't some unmanned system somewhere doing more complex reasoning. There was a talk years ago at IJCAI from some people from NASA Ames on a prototype aircraft-control system they'd built that used a reasoning system to assist with performing emergency landings in situations with no preprogrammed contingency, by taking into account some telemetry information (e.g. aircraft damage), map information, an aerodynamic model, and risk models. I do believe they were planning to deploy it as a suggestion system though, which would suggest a course of action, and then leave it to the pilot to implement it or not. Then the judgment gets more murky; now the system is doing some of the judgment (evaluation of alternatives, etc.) that a human pilot would normally do, but leaving some of the judgment (accept the suggestion, modify it) still to the human. edit: Here's a more recent paper than the one I'm thinking of, but must be the same project: http://ti.arc.nasa.gov/m/profile/de2smith/publications/IAAI09-ELP.pdf http://ti.arc.nasa.gov/m/profile/de2smith/publications/IAAI0...

onemoreact 15y ago

That's not really true. It's often the case that automation is designed reach a specific goal and it then try's to achieve / maintain that state. AKA Segways try to balance and an F-15 Control Augmentation System (CAS) aka stability assistance system try's to keep flying even without a wing. (Yes, this worked and was not programmed for.) http://www.airliners.net/aviation-forums/tech_ops/print.main?id=84282 http://www.airliners.net/aviation-forums/tech_ops/print.main... Plenty of other planes of lost a section of wing a wing and still landed. http://www.airliners.net/aviation-forums/general_aviation/read.main/1252625/ http://www.airliners.net/aviation-forums/general_aviation/re... Granted, all of these cases had a pilot, but in the F-15 the avionics actually discovered how to maintain level flight after the loss of the wing.

lucasjung 15y ago
I have a master's in aeronautical engineering, and one of my major areas of study was stability and control, which is what your Segway and F-15 examples fall under. Automated flight control systems definitely do not exercise "judgement:" they have inputs, a transfer function (typically MIMO, these days), and outputs. It used to be that the transfer function was fixed, but more sophisticated systems (e.g. fighter jets) often have many different transfer functions and switch between them based upon various inputs. They don't "decide" or "discover" anything: for any given set of inputs, they will predictably produce a pre-determined set of outputs. Aircraft stability and control systems are not programmed to care about, or even know about, the existance of the wings. The closest they get to this is that they will know the current states of the control surfaces on the wings. So it doesn't really make sense to say that the F-15 CAS was "not programmed for" the state of missing a wing (although it almost certainly was programmed to respond properly in the situation where it gets no feedback from some of the flight controls). As you say, it's designed to reach a specific goal (keep the plane level) and then maintain that state. If the system detects an uncommanded roll-rate, it will move the flight controls to stop that roll-rate. It doesn't know or care that the uncommanded roll-rate is the result of asymmetric lift due to an (almost completely) missing wing: it's just going to keep moving the flight control surfaces until that roll rate goes away. If the aicraft had been damaged in a slightly different way, it's possible that the CAS would have issued commands to the flight controls that would have departed the plane, but fortunately the handling characteristics of the aircraft remained close enough to normal that the control laws still produced good results. Unfortunately, this behavior can result in mishaps when a flight control system gets erroneous inputs: when it believes that it is rolling when it is wings-level (or believies it is level when it is rolling). This was a major contributor to the Air France fligh 447 crash. In such situations, it takes judgement to realize that something is wrong and to figure out what to do about it.

rbanffy 15y ago

Before claiming a machine is incapable of judgment, we would have to define what it is. Is a fly capable of judgment? A fish? This is a philosophical question and won't yield any meaningful answers. Call it judgment of feedback loop, the end result is similar.