4 ms·

> but you now think the algorithms that might get you okay results on Kaggle competitions can be used to predict a child's future crimes Do you think anything

> but you now think the algorithms that might get you okay results on Kaggle competitions can be used to predict a child's future crimes Do you think anything can be used to predict crimes? Can you imagine any fair application of justice that doesn't boil down to an algorithm? Sentencing guidelines are an algorithm. It's literally a table on page 3[0]. Do you think sentencing guidelines, which were invented to address unfairness in the justice system like this guy's work was, are unfair? What alternative do you propose? Case-by-case judgements? Even if you could find me a model judge, be honest: the judge probably uses something that boils down to an algorithm to be just. > brilliant proxy for race Suppose we only used variables with a low correlation to race. Or we decorrelate them from race somehow. Suppose error on the validation set increased. Would this be sufficient to make this system more just to you? Would a reasonable person think it is more just? > Specific ML systems ("deep") can infer hidden variables such as race Considering how scientific you are trying to be about your understanding of ML, you're being dishonest about how much ML systems can "infer hidden variables." There's strictly anecdotal evidence of "deep"ness inferring something hidden. We think GoogLeNet is seeing higher-order features, but then we show it a picture of a cat and it thinks it's looking at a paper towel.[1] You're overstating the power of these systems to serve your point, ironically exactly what you accuse this guy of doing. > Resume screening, credit card applications, parole risk classification... potential misuses I can see how to an individual these things are unfair. But how about in aggregate? If I can show you a way to process credit card applications that makes more profit, reduces bankruptcies and increases credit usage, would you say, "don't use it"? So what if it's made with ML, or pixie dust? Would you reject these things? > destructive feedback loop That's just society dude :) [0] http://www.ussc.gov/sites/default/files/pdf/about/overview/Overview_Federal_Sentencing_Guidelines.pdf http://www.ussc.gov/sites/default/files/pdf/about/overview/O... [1] https://codewords.recurse.com/issues/five/why-do-neural-networks-think-a-panda-is-a-vulture https://codewords.recurse.com/issues/five/why-do-neural-netw...

Smerity 10y ago
The issue is not whether justice can be codified. Let us assume that you can and it's outlined on page 3 of your document. Providing examples to an ML model don't guarantee it will codify anything similar to that. This is a fundamental issue. Most machine learning models aren't highly interpretable and can also act in unexpected ways. Re: (a) variables with a low correlation to race / (b) dishonest about how much ML systems can "infer hidden variables": (b) The example you point to is specifically adversarial modification to the input data. I'm not sure how that's relevant to your point at hand except to say that ML models can be misguided when given slightly modified input. It doesn't show that inferring hidden variables is anecdotal. If you look at the work of machine translation with attention, it explicitly learns how to do alignments even though it is never given training data for it. We can visualize that[1]. (a) There is interesting work on trying to produce variables with a low correlation to race. One recent example[2] has two networks use the same intermediate representation created from the raw input. One of the networks tries to identify a particular variable (race / gender / ...) while the other tries to perform the overall objective task. We aim to modify the way we generate the intermediate representation such that the former network has a harder time identifying the sub groups whilst allowing the latter group to continue to improve on the original task. This is recent however. There are many other interesting explorations. Regarding when to use features that may leak information, this is referred to as the fairness / utility tradeoff. If you want to see an example of both that and trying to prevent information leakage about gender in a dataset on adult income, refer to "Certifying and removing disparate impact"[3]. Note: the models used by the statistician in the article do not appear to feature anywhere near this level of analysis. Ensuring to avoid features that result in strong bias or prejudice against specific groups in a dataset is still an open problem. [1]: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2015/07/Figure6_sample_translations1-624x282.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... (image from https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/ https://devblogs.nvidia.com/parallelforall/introduction-neur...) [2]: http://arxiv.org/abs/1511.05897 http://arxiv.org/abs/1511.05897 [3]: http://arxiv.org/abs/1412.3756 http://arxiv.org/abs/1412.3756

Smerity 10y ago

The issue is not whether justice can be codified. Let us assume that you can and it's outlined on page 3 of your document. Providing examples to an ML model don't guarantee it will codify anything similar to that. This is a fundamental issue. Most machine learning models aren't highly interpretable and can also act in unexpected ways. Re: (a) variables with a low correlation to race / (b) dishonest about how much ML systems can "infer hidden variables": (b) The example you point to is specifically adversarial modification to the input data. I'm not sure how that's relevant to your point at hand except to say that ML models can be misguided when given slightly modified input. It doesn't show that inferring hidden variables is anecdotal. If you look at the work of machine translation with attention, it explicitly learns how to do alignments even though it is never given training data for it. We can visualize that[1]. (a) There is interesting work on trying to produce variables with a low correlation to race. One recent example[2] has two networks use the same intermediate representation created from the raw input. One of the networks tries to identify a particular variable (race / gender / ...) while the other tries to perform the overall objective task. We aim to modify the way we generate the intermediate representation such that the former network has a harder time identifying the sub groups whilst allowing the latter group to continue to improve on the original task. This is recent however. There are many other interesting explorations. Regarding when to use features that may leak information, this is referred to as the fairness / utility tradeoff. If you want to see an example of both that and trying to prevent information leakage about gender in a dataset on adult income, refer to "Certifying and removing disparate impact"[3]. Note: the models used by the statistician in the article do not appear to feature anywhere near this level of analysis. Ensuring to avoid features that result in strong bias or prejudice against specific groups in a dataset is still an open problem. [1]: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2015/07/Figure6_sample_translations1-624x282.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... (image from https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/ https://devblogs.nvidia.com/parallelforall/introduction-neur...) [2]: http://arxiv.org/abs/1511.05897 http://arxiv.org/abs/1511.05897 [3]: http://arxiv.org/abs/1412.3756 http://arxiv.org/abs/1412.3756