3 ms·
> with a motivated choice of training data Intent is not required. > only going to have evidence-based biases That includes historic institutional biases. Fo
by pdkl95 9y ago
> with a motivated choice of training data
Intent is not required.
> only going to have evidence-based biases
That includes historic institutional biases. For example, many US cities are still highly segregated by race. This was done intentionally as a workaround for forced-integration rulings with indirect methods such as redlining, blockbusting, and restrictive covenants. While most of these methods were eventually banned, the results of those methods still exist today[1].
Since school funding usually depends on property taxes, these policies also impacted the school system. It may be a fact that one group of people performs worse on some metric, but how much of that was caused by racist housing practices decades earlier? Unless you're accounting for all of the embedded biases already present in the facts, you are laundering those biases into your results.
[1] https://www.huffingtonpost.com/entry/the-9-most-segregated-cities-in-america_us_55df53e9e4b0e7117ba92d7f https://www.huffingtonpost.com/entry/the-9-most-segregated-c...
- closeparen 9y agoPeople building classifiers are not typically trying to make a causal argument, they just want the classifier.
- pdkl95 9y agoClassifying anything related to people is a political act; it isn't possible to be apolitical and "just make a classifier". The classifier either accounts for the biases already present in the data, or they are present in the output, obscured (laundered) behind an opaque wall of machine learning.
- closeparen 9y agoUnjust policies laundered those biases into reality, at which point they stop being biases and start being facts with unjust causes. Whitewashing the training data to make predictions based on a hypothetical injustice-free world where redlining never happened will just give you garbage predictions.
- pdkl95 9y agoI agree that it's hard to get good predictions. The point, however, is that ignoring systemic bias and other historic influences is worse. > and start being facts with unjust causes. Of course. That means you know the data contains an "unjust" bias. Do you want to account for that bias, or do you want to proceed anyway risking problems (e.g. Simpson's Paradox). There are more confounding factors in heaven and earth, closeparen, than are dreamt of in your model. https://mathbabe.org/2017/01/04/recidivism-risk-algorithms-are-inherently-discriminatory/ https://mathbabe.org/2017/01/04/recidivism-risk-algorithms-a... https://mathbabe.org/2013/01/29/bill-gates-is-naive-data-is-not-objective/ https://mathbabe.org/2013/01/29/bill-gates-is-naive-data-is-...
- closeparen 9y agoConfounding factors undermine causal arguments. You can build a perfectly useful model without knowing about the hidden variables. A risk-management approach to the criminal justice system doesn’t call for fudge factors. It’s fundamentally unacceptable. Risk management is all about using people’s demographics against them (see: all insurance pricing). Wokeness on the part of the risk managers doesn’t fix that. The criminal justice system needs to treat people as individuals and not classes - this requires that it steer clear of evidence-based risk assessment.