3 ms·

The question projects moralism onto the process of justice; the word “guilt” perhaps carries with it a demand for a sentient being making moral choices. Howeve

by sunir 7y ago

The question projects moralism onto the process of justice; the word “guilt” perhaps carries with it a demand for a sentient being making moral choices. However what is important is what has to be done to protect society. The question is does the system merit exile or destruction or some other remediation to restore order. The answer is based on evidence and observed behaviour... yes. HAL should be decommissioned. It is exactly similar to the question “is the 737MAX guilty of murder?” Guilt and murder are concepts related to conscious thought but the behaviour and outcomes is what really matters in that case. The reason why guilt and murder matter is as humans, we have instincts around managing each other that these concepts extend with the power of Reason and Process. If it was a different species such as a sentient ant colony, could we project the concepts of guilt and murder equivalently? Probably not.

pjc50 7y ago
The reason why most legal systems distinguish various grades of causing unlawful death is precisely because intent matters. https://en.wikipedia.org/wiki/Manslaughter https://en.wikipedia.org/wiki/Manslaughter Mapping this onto complex systems is hard, but usually we presume that the system is under the control of the designer and therefore the intent or carelessness of the designer is what matters. AI systems offer the disconcerting possibility that at some point the designers will start explicitly disclaiming fore-knowledge and therefore responsibility for what the system is doing. Nontransparent neural networks are a particularly awkward problem here.
- dschuetz 7y ago
  > AI systems offer the disconcerting possibility that at some point the designers will start explicitly disclaiming fore-knowledge and therefore responsibility for what the system is doing. Nontransparent neural networks are a particularly awkward problem here. Exactly my point. When there is a black-boxed AI involved an engineer (as of now) cannot precisely determine whether it functions properly (according to specifications, from which safety rules are derived) or not. So, the monitoring technology is not as advanced as it should be to let black-boxed AIs handle important functions. So, until any AI is not as competent as we are, we should not employ black-boxed AI in algorithms at all, because it is irresponsible (before the law, as consequence). We cannot say it (the AI) is responsible for odd behavior, when we programmed/trained it that way. Same goes for children, parents are responsible for their children. Gross negligence (like letting the kid skipping school repeatedly) is punishable before the law, and parents are responsible, not the child.
  - AstralStorm 7y ago
    Parents are responsible to a extent they could have predicted the actions of the child. There are many cases where parents are clearly not held responsible by law, including outright cases of murder. (Usually considered as manslaughter with equivalent of insanity defense.) Unless you are of the position that everyone is guilty, by induction. In case of HAL, the negligence case is clear because there existed a perfect simulator - SAL. SAL was not given the secret orders, which is clearly endangering the crew. Whoever is responsible for that difference clearly knew that changing the orders made the simulation invalid. It's similar to Boeing 737 MAX case.
    - dschuetz 7y ago
      I agree, parents do not have 'total' control over their kids, but they have to ensure their well-being, get medical care in cases of illness (physical, mental). If they neglect that responsibility, they are as equally responsible for any outcome, "your mileage might vary" depending what laws your state has. Same goes for engineers, they cannot completely control the durability of, say, steel, hence they cannot be held accountable for bad quality of steel (except for poor judgement buying cheap/bad steel). But someone else is responsible for quality. And so on. So, yes, with untested scenarios (like "secret orders") responsibility fall into the hands of those who imposed the orders and omitted crucial tests. It's not even the engineers fault anymore.
- sunir 7y ago
  Intent is a method of predicting the probability of repeat offense on one hand and on the other hand the magnitude of sin. Sin and moralism is an instinctive feeling that presumably corresponds to risk management. With an AI, intent (meaning reasoning or modelling) is also measurable. So taking 737MAX again, because the software will reliably and consistently lethally fail, all planes globally were grounded. If it was the case of metal fatigue in one aileron then the entire worldwide fleet would less likely be grounded.

by sunir 7y ago

pjc50 7y ago

The reason why most legal systems distinguish various grades of causing unlawful death is precisely because intent matters. https://en.wikipedia.org/wiki/Manslaughter https://en.wikipedia.org/wiki/Manslaughter Mapping this onto complex systems is hard, but usually we presume that the system is under the control of the designer and therefore the intent or carelessness of the designer is what matters. AI systems offer the disconcerting possibility that at some point the designers will start explicitly disclaiming fore-knowledge and therefore responsibility for what the system is doing. Nontransparent neural networks are a particularly awkward problem here.

dschuetz 7y ago
> AI systems offer the disconcerting possibility that at some point the designers will start explicitly disclaiming fore-knowledge and therefore responsibility for what the system is doing. Nontransparent neural networks are a particularly awkward problem here. Exactly my point. When there is a black-boxed AI involved an engineer (as of now) cannot precisely determine whether it functions properly (according to specifications, from which safety rules are derived) or not. So, the monitoring technology is not as advanced as it should be to let black-boxed AIs handle important functions. So, until any AI is not as competent as we are, we should not employ black-boxed AI in algorithms at all, because it is irresponsible (before the law, as consequence). We cannot say it (the AI) is responsible for odd behavior, when we programmed/trained it that way. Same goes for children, parents are responsible for their children. Gross negligence (like letting the kid skipping school repeatedly) is punishable before the law, and parents are responsible, not the child.
- AstralStorm 7y ago
  Parents are responsible to a extent they could have predicted the actions of the child. There are many cases where parents are clearly not held responsible by law, including outright cases of murder. (Usually considered as manslaughter with equivalent of insanity defense.) Unless you are of the position that everyone is guilty, by induction. In case of HAL, the negligence case is clear because there existed a perfect simulator - SAL. SAL was not given the secret orders, which is clearly endangering the crew. Whoever is responsible for that difference clearly knew that changing the orders made the simulation invalid. It's similar to Boeing 737 MAX case.
  - dschuetz 7y ago
    I agree, parents do not have 'total' control over their kids, but they have to ensure their well-being, get medical care in cases of illness (physical, mental). If they neglect that responsibility, they are as equally responsible for any outcome, "your mileage might vary" depending what laws your state has. Same goes for engineers, they cannot completely control the durability of, say, steel, hence they cannot be held accountable for bad quality of steel (except for poor judgement buying cheap/bad steel). But someone else is responsible for quality. And so on. So, yes, with untested scenarios (like "secret orders") responsibility fall into the hands of those who imposed the orders and omitted crucial tests. It's not even the engineers fault anymore.
sunir 7y ago
Intent is a method of predicting the probability of repeat offense on one hand and on the other hand the magnitude of sin. Sin and moralism is an instinctive feeling that presumably corresponds to risk management. With an AI, intent (meaning reasoning or modelling) is also measurable. So taking 737MAX again, because the software will reliably and consistently lethally fail, all planes globally were grounded. If it was the case of metal fatigue in one aileron then the entire worldwide fleet would less likely be grounded.