4 ms·

If no system of rules can ever cover the policy space, then the policy itself is necessarily ill-defined. That alone is a problem before automation considerati

If no system of rules can ever cover the policy space, then the policy itself is necessarily ill-defined. That alone is a problem before automation considerations. I'm arguing that automation provides the opportunity to make good, open assessment part of a process of improvement rather than opaquely driving firings.

brundolf 5y ago
> then the policy itself is necessarily ill-defined I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. The conceit that it could is one of the biggest driving factors of our nascent tech-dystopia. An open policy would of course be better than a closed policy, if only because it would lay bare all of its flaws, but even then it could never be completely fixed.
- neuronexmachina 5y ago
  > I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. "All models are wrong, but some are useful."
  - version_five 5y ago
    Not when people are punished as a result of them. It's fine to make a generalization that works on average for some low stakes, benign decision, but when it materially affects real people, it is not right to generalize. This is the core argument imo against algorithmic decision making of this kind.
    - julienb_sea 5y ago
      You can make the exact same argument in reverse. When decisions materially affect real people, it is unacceptable and fundamentally unfair to rely on human judgement, along with its biases and capriciousness. Clear, transparent policy applied evenly and without bias is a more fair approach for people.
      - version_five 5y ago
        No, it doesnt work that way. By projecting real, complex decisions down to some low dimensional set of criteria, you automatically throw away information and generalize. It's only fair in the sense that the criteria is the same, but it's necessarily based on some kind of average outcome and so effectively sacrifices outliers. Its is still extremely unfair to inputs (who are actual people in the cases we mean) that exhibit characteristics that reliably fit the prediction on average, but that don't conform to the prediction.
hnick 5y ago
This seems like a restatement of the argument that if we just code enough 'if..then..else' statements, we'd get a well-behaved general AI. That approach hasn't worked, and complicating rules and procedures to cover all edge cases will fall short too. That's why judges are still humans. The list of edge cases, the number of factors that can go into a decision, is correlated to the number of things possible in reality. It's basically unbounded. If I've misunderstood and you are arguing instead for a clear and minimal set of rules to flag and trigger a human review, that sounds sensible to me. In my experience though when companies find an exception they update the policy, and before you know it it becomes a tangled mess.
- kjhughes 5y ago
  Such a hybrid, automanual system is certainly viable, and your analogy to laws being rules and judges being interpreters is reasonable. In that context, consider that a single law, "do the right thing," might require too many judges and might produce too inconsistent of outcomes to work well. "The right thing" may have to be better defined. Similarly for company policy, it is not the automation used to express bad, ill-defined, or secretive policy that's the problem. The problem is the policy itself.

brundolf 5y ago

> then the policy itself is necessarily ill-defined I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. The conceit that it could is one of the biggest driving factors of our nascent tech-dystopia. An open policy would of course be better than a closed policy, if only because it would lay bare all of its flaws, but even then it could never be completely fixed.

neuronexmachina 5y ago
> I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. "All models are wrong, but some are useful."
- version_five 5y ago
  Not when people are punished as a result of them. It's fine to make a generalization that works on average for some low stakes, benign decision, but when it materially affects real people, it is not right to generalize. This is the core argument imo against algorithmic decision making of this kind.
  - julienb_sea 5y ago
    You can make the exact same argument in reverse. When decisions materially affect real people, it is unacceptable and fundamentally unfair to rely on human judgement, along with its biases and capriciousness. Clear, transparent policy applied evenly and without bias is a more fair approach for people.
    - version_five 5y ago
      No, it doesnt work that way. By projecting real, complex decisions down to some low dimensional set of criteria, you automatically throw away information and generalize. It's only fair in the sense that the criteria is the same, but it's necessarily based on some kind of average outcome and so effectively sacrifices outliers. Its is still extremely unfair to inputs (who are actual people in the cases we mean) that exhibit characteristics that reliably fit the prediction on average, but that don't conform to the prediction.

hnick 5y ago

This seems like a restatement of the argument that if we just code enough 'if..then..else' statements, we'd get a well-behaved general AI. That approach hasn't worked, and complicating rules and procedures to cover all edge cases will fall short too. That's why judges are still humans. The list of edge cases, the number of factors that can go into a decision, is correlated to the number of things possible in reality. It's basically unbounded. If I've misunderstood and you are arguing instead for a clear and minimal set of rules to flag and trigger a human review, that sounds sensible to me. In my experience though when companies find an exception they update the policy, and before you know it it becomes a tangled mess.

kjhughes 5y ago
Such a hybrid, automanual system is certainly viable, and your analogy to laws being rules and judges being interpreters is reasonable. In that context, consider that a single law, "do the right thing," might require too many judges and might produce too inconsistent of outcomes to work well. "The right thing" may have to be better defined. Similarly for company policy, it is not the automation used to express bad, ill-defined, or secretive policy that's the problem. The problem is the policy itself.