5 ms·

It seems like the problem is that any data that works statistically will not be fair to individuals who are intuitively exceptions to the general trend. It soun

It seems like the problem is that any data that works statistically will not be fair to individuals who are intuitively exceptions to the general trend. It sounds like algorithms relying on statistics are pretty much guaranteed to fail the fairness test. It's the wrong tool for this problem I think. There are so many factors in morality that I think individually, case by case judgments are likely the best option. Racial / class bias seems to be just as present in these algorithms, so lets at least allow for human intervention rather than forcing ourselves to be constrained by an inherently unfair system. Yes, humans are flawed too, but perhpas we should be spending more time trying to adjust / account for those in different ways. Note: Bias in sentencing/convictions is a huge problem that should absolutely be worked on. I'm only claiming that these approaches are inherently flawed, not that what we have now is anywhere close to good enough.

quotemstr 9y ago
> There are so many factors in morality that I think individually, case by case judgments are likely the best option. What are the inputs to "case by case judgments" except Bayesian priors and observations about a specific case? Your brain isn't doing anything a computer model can't in principle do. My point isn't that any particular bail pricing model is acceptable. Instead, I'm arguing that a desire to escape "statistics" by giving up on models and punting to the brain is futile, since the brain is just going to use its own statistical model whether the owner of that brain is aware of it or not --- it must, since statistics is baked into the structure of knowledge, and the brain has no private source of truth. There is no accessible realm of knowledge about the real world somehow exempt from statistics, since we never have complete knowledge, and statistics models our uncertainty.
- adrianN 9y ago
  While I agree with your general point, artificial neural networks and biological neural networks are sufficiently different that it's not really fair to compare them just because they happen to have the same name. A biological neuron does a lot more than computing some affine transformation of its inputs. Also, humans draw from an absurdly complex pool of inputs, usually called "common knowledge", that so far has given us a hard time when trying to replicate it in computer systems. So while in principle there is, imho, nothing stopping a statistical method to produce decisions at least as good as a human could, in practice this might not be the case.
  - noobermin 9y ago
    To expand on your point to further strengthen your original argument, these models are trained on a smaller subset of observables and ideas we consider when we decide policy. Even then, the comparison between a matrix and a human brain is poor. Another issue I have is that we don't know is whether statistical bias exists in the metrics they train on, something they discuss in the paper. We already know for example blacks are arrested and convicted harsher than whites for some of the similar crimes[0]. The paper just says that COMPAS predicts likelihood of violent crime, not whether that means they'll be arrested of crime. Moreover while Table 1 talks about a prediction regarding violent crime, Figure 2 talks about recidivism with regards to all crime, so which is it? It still seems nebulous to me what COMPAS actually is for. I'd have to read it deeper than I have, but not sure how I feel about the paper. [0] http://www.politifact.com/truth-o-meter/statements/2016/feb/26/hillary-clinton/hillary-clinton-says-blacks-more-likely-be-arreste/ http://www.politifact.com/truth-o-meter/statements/2016/feb/...
- dblotsky 9y ago
  1. This article doesn't mention neural networks. 2. "Your brain isn't doing anything a neural network can't do." Really, now? 3. The paper mentions "over 100 factors" the algorithm uses. Ad agencies have way more than 100 factors on me, and I still get irrelevant ads every day. Humans aren't perfect, but they can catch things that stats won't, and vice versa.
  - quotemstr 9y ago
    > 2. "Your brain isn't doing anything a neural network can't do." Really, now? They're both Turing machine approximators after all. Granted, ANNs aren't quite sophisticated enough to match the brain yet, but this capability gap is a difference of degree, not kind. But I don't think the similarity of the brain to ANNs really matters to my argument, as interesting as this discussion might be --- so I've edited my original comment to make it more general and to focus on my higher-level point.
    - deleted 9y ago
      [deleted]
    - dblotsky 9y ago
      The difference between mathematical neural networks and humans is how they're trained and how they function, which is not just a difference in kind. The way that they adjust weights and perform inference is different enough that them being "theoretically isomorphic" is not a viable argument.
- dblotsky 9y ago
  The algorithm also does "case by case" judgments. However its judgments are purely based on conditional probabilities observed from past events, which are computed using regression analysis (a relatively simple method). Human learning is enormously different from regression analysis. > I'm arguing that a desire to escape "statistics" by giving up on models and punting to the brain is futile I think we'd all agree here. The point is that both methods should always be used together, because each has huge limitations on its own.
  - quotemstr 9y ago
    What is the brain using except conditional probabilities derived from past inputs? Why would the use of some kind of regression (which I wouldn't characterize as "relatively simple") a problem if the overall result is nevertheless more accurate than the human equivalent? We judge models by their results, not their complexity. The brain is applying some function over its inputs, after all, and we can apply that same function, either explicitly in code or implicitly via the training of some kind of model. Machines are already better than humans at other tasks for which we imagined humans exercised fine and subtle judgment: https://www.sciencedaily.com/releases/2016/04/160421133831.htm https://www.sciencedaily.com/releases/2016/04/160421133831.h..., so I see no reason to believe that bail-setting is some unapproachable problem domain. > The point is that both methods should always be usned together, because each has huge limitations on its own. I disagree. Putting a human in the loop just invites those humans to inject unstated, socially-determined biases into the decision system, and fairness and transparency dictate that we be up-front about those biases. I see no evidence that a computer model couldn't reproduce biases we considered desirable.
    - dblotsky 9y ago
      > What is the brain using except conditional probabilities derived from past inputs? Hormones, reflexes, instincts, empathy, etc. Again, human inference and learning are (as of yet) insurmountably more complex than statistical regression. > regression (which I wouldn't characterize as "relatively simple" I encourage you to study brain chemistry, and compare that with any stats course ever. > Why would the use of some kind of regression (which I wouldn't characterize as "relatively simple") a problem if the overall result is nevertheless more accurate than the human equivalent? We judge models by their results, not their complexity. Because there is no way to universally judge "model results". https://en.wikipedia.org/wiki/Confusion_matrix https://en.wikipedia.org/wiki/Confusion_matrix Even a 99% true positive rate + 5% false positive rate can be worse than a 5% true positive rate and a 0% false positive rate, when you're talking about irreversible decisions. > The brain is applying some function over its inputs, after all, and we can apply that same function, either explicitly in code or implicitly via the training of some kind of model. Yes, that's very true. Executing the model is easy, and maybe even works similarly in NNs and brains. Figuring out that "some kind" part is what we formalise as "learning", and turns out is the hardest part of this equation. Again, clearly, current regression methods are nothing like human learning. Else, they'd be able to learn a language like a human, without seeing billions of annotated data points. > Machines are already better than humans at ... Neither the article nor the paper claim the machine performs better. Their strong claim is that the machine is faster. > I see no reason to believe that bail-setting is some unapproachable problem domain. I think justice is a more important domain for many people. My guess is it's because it can set precedent. > Putting a human in the loop just invites those humans to inject unstated, socially-determined biases into the decision system For drastic contrast: unstated, socially-determined biases is what prevents nuclear wars. https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov More plainly: people have compassion, empathy, and can understand things about convicts that are (still) very difficult to encode into a computer.

quotemstr 9y ago

> There are so many factors in morality that I think individually, case by case judgments are likely the best option. What are the inputs to "case by case judgments" except Bayesian priors and observations about a specific case? Your brain isn't doing anything a computer model can't in principle do. My point isn't that any particular bail pricing model is acceptable. Instead, I'm arguing that a desire to escape "statistics" by giving up on models and punting to the brain is futile, since the brain is just going to use its own statistical model whether the owner of that brain is aware of it or not --- it must, since statistics is baked into the structure of knowledge, and the brain has no private source of truth. There is no accessible realm of knowledge about the real world somehow exempt from statistics, since we never have complete knowledge, and statistics models our uncertainty.

adrianN 9y ago
While I agree with your general point, artificial neural networks and biological neural networks are sufficiently different that it's not really fair to compare them just because they happen to have the same name. A biological neuron does a lot more than computing some affine transformation of its inputs. Also, humans draw from an absurdly complex pool of inputs, usually called "common knowledge", that so far has given us a hard time when trying to replicate it in computer systems. So while in principle there is, imho, nothing stopping a statistical method to produce decisions at least as good as a human could, in practice this might not be the case.
- noobermin 9y ago
  To expand on your point to further strengthen your original argument, these models are trained on a smaller subset of observables and ideas we consider when we decide policy. Even then, the comparison between a matrix and a human brain is poor. Another issue I have is that we don't know is whether statistical bias exists in the metrics they train on, something they discuss in the paper. We already know for example blacks are arrested and convicted harsher than whites for some of the similar crimes[0]. The paper just says that COMPAS predicts likelihood of violent crime, not whether that means they'll be arrested of crime. Moreover while Table 1 talks about a prediction regarding violent crime, Figure 2 talks about recidivism with regards to all crime, so which is it? It still seems nebulous to me what COMPAS actually is for. I'd have to read it deeper than I have, but not sure how I feel about the paper. [0] http://www.politifact.com/truth-o-meter/statements/2016/feb/26/hillary-clinton/hillary-clinton-says-blacks-more-likely-be-arreste/ http://www.politifact.com/truth-o-meter/statements/2016/feb/...
dblotsky 9y ago
1. This article doesn't mention neural networks. 2. "Your brain isn't doing anything a neural network can't do." Really, now? 3. The paper mentions "over 100 factors" the algorithm uses. Ad agencies have way more than 100 factors on me, and I still get irrelevant ads every day. Humans aren't perfect, but they can catch things that stats won't, and vice versa.
- quotemstr 9y ago
  > 2. "Your brain isn't doing anything a neural network can't do." Really, now? They're both Turing machine approximators after all. Granted, ANNs aren't quite sophisticated enough to match the brain yet, but this capability gap is a difference of degree, not kind. But I don't think the similarity of the brain to ANNs really matters to my argument, as interesting as this discussion might be --- so I've edited my original comment to make it more general and to focus on my higher-level point.
  - deleted 9y ago
    [deleted]
  - dblotsky 9y ago
    The difference between mathematical neural networks and humans is how they're trained and how they function, which is not just a difference in kind. The way that they adjust weights and perform inference is different enough that them being "theoretically isomorphic" is not a viable argument.
dblotsky 9y ago
The algorithm also does "case by case" judgments. However its judgments are purely based on conditional probabilities observed from past events, which are computed using regression analysis (a relatively simple method). Human learning is enormously different from regression analysis. > I'm arguing that a desire to escape "statistics" by giving up on models and punting to the brain is futile I think we'd all agree here. The point is that both methods should always be used together, because each has huge limitations on its own.
- quotemstr 9y ago
  What is the brain using except conditional probabilities derived from past inputs? Why would the use of some kind of regression (which I wouldn't characterize as "relatively simple") a problem if the overall result is nevertheless more accurate than the human equivalent? We judge models by their results, not their complexity. The brain is applying some function over its inputs, after all, and we can apply that same function, either explicitly in code or implicitly via the training of some kind of model. Machines are already better than humans at other tasks for which we imagined humans exercised fine and subtle judgment: https://www.sciencedaily.com/releases/2016/04/160421133831.htm https://www.sciencedaily.com/releases/2016/04/160421133831.h..., so I see no reason to believe that bail-setting is some unapproachable problem domain. > The point is that both methods should always be usned together, because each has huge limitations on its own. I disagree. Putting a human in the loop just invites those humans to inject unstated, socially-determined biases into the decision system, and fairness and transparency dictate that we be up-front about those biases. I see no evidence that a computer model couldn't reproduce biases we considered desirable.
  - dblotsky 9y ago
    > What is the brain using except conditional probabilities derived from past inputs? Hormones, reflexes, instincts, empathy, etc. Again, human inference and learning are (as of yet) insurmountably more complex than statistical regression. > regression (which I wouldn't characterize as "relatively simple" I encourage you to study brain chemistry, and compare that with any stats course ever. > Why would the use of some kind of regression (which I wouldn't characterize as "relatively simple") a problem if the overall result is nevertheless more accurate than the human equivalent? We judge models by their results, not their complexity. Because there is no way to universally judge "model results". https://en.wikipedia.org/wiki/Confusion_matrix https://en.wikipedia.org/wiki/Confusion_matrix Even a 99% true positive rate + 5% false positive rate can be worse than a 5% true positive rate and a 0% false positive rate, when you're talking about irreversible decisions. > The brain is applying some function over its inputs, after all, and we can apply that same function, either explicitly in code or implicitly via the training of some kind of model. Yes, that's very true. Executing the model is easy, and maybe even works similarly in NNs and brains. Figuring out that "some kind" part is what we formalise as "learning", and turns out is the hardest part of this equation. Again, clearly, current regression methods are nothing like human learning. Else, they'd be able to learn a language like a human, without seeing billions of annotated data points. > Machines are already better than humans at ... Neither the article nor the paper claim the machine performs better. Their strong claim is that the machine is faster. > I see no reason to believe that bail-setting is some unapproachable problem domain. I think justice is a more important domain for many people. My guess is it's because it can set precedent. > Putting a human in the loop just invites those humans to inject unstated, socially-determined biases into the decision system For drastic contrast: unstated, socially-determined biases is what prevents nuclear wars. https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov More plainly: people have compassion, empathy, and can understand things about convicts that are (still) very difficult to encode into a computer.