2 ms·
Huh? It seems to me that the OP is using the example of race correlating with zip code as another example of how data can be misused and misinterpreted, e.g. fe
by danso 9y ago
Huh? It seems to me that the OP is using the example of race correlating with zip code as another example of how data can be misused and misinterpreted, e.g. feeding a dataset that has zip code and income could still lead to an AI making a model not much different than if the data had race in it.
- TeMPOraL 9y agoI believe the article implies that calling it "misuse" or "misinterpretation" is preferring "polite fiction" to reality.
- danso 9y agoThe text does not support your interpretation. He says that data in the United States almost always has a racial bias, and that even when you explicitly remove race from the data, you and your ML model can still inadvertently make predictions that fall along racial lines because of how other data points strongly correlate to race. He then goes on to talk about COMPAS, which was accused of being racially biased even though COMPAS explicitly did not include race as one of its inputs. The author argues that COMPAS still mad predictions that were racially biased because of how strong the correlation is between race and housing. On top of that, the author argues, COMPAS asked the wrong question (who is likely to be more convicted, vs who is more likely to commit crime). The author does not, as you seem to do, celebrate the Justice system algorithms for making harsher, but objective data-driven recommendations for minority groups. He argues that this is unjust, that the AI is missing the necessary nuance and context, and that the courts should not have been so trusting of AI: > There are obviously many problems at play here. One is that the courts took the AI model far too seriously, using it as a direct factor in sentencing decisions, skipping human judgment, with far more confidence than any model should warrant. (A good rule of thumb, also recently encoded into EU law, is that decisions with serious consequences of people should be sanity-checked by a human — and that there should be a human override mechanism available.) Another problem, of course, is the underlying systemic racism which this exposed: the fact that Black people are more likely to be arrested and convicted of the same crimes.