4 ms·
In the case you presented, the biases are in the dataset itself. Arrest statistics in the US are heavily skewed by race. If you were to take a dataset of all U
by woodruffw 10y ago
In the case you presented, the biases are in the dataset itself.
Arrest statistics in the US are heavily skewed by race. If you were to take a dataset of all US arrests between 1900 and 2000 and ask which populations are most likely to "commit crimes" (i.e., be arrested), you'd get racially biased model without recording why it's biased (discrimination, enduring poverty, minimum sentencing, &c).
- dante821 10y agoCan't that problem be solved by simply not telling the machine the race of the person who's being judged?
- mattnewton 10y agoDead sibling comment asks a common questions I hear: why can't we just not tell the algorithm race? And the answer is that, with enough data points, if there is a statistical bias in the dataset, the algorithm will likely learn a substitute for race in the other features. For the sake of argument, perhaps an interaction between home address, annual income and model of car driven is predictive of race, and race is predictive of recidivism in the dataset- then the algorithm will learn this cross of features is predictive of recividism, even though we would like all of them to ideally be irrelevant to sentencing.
- gravypod 10y ago> home address, annual income and model of car driven is predictive Why the hell would those be factors? The factors of the case should be things that actually matter in the case. A rich person who murders someone should get the same sentence as a poor person who murders someone. A black person who murders someone should get the same sentence as a white person who murders someone. Adding those factors would be insane in the first place. If you're adding crazy things like that you might as well add factors like "Can Juggle" and "Can Burp the Alphabet" because things like that should have just as much to do with sentencing as what kind of car you drive or where you live.
- jcranmer 10y agoOne example that's going to come up is "Has the person previously been convicted of a crime?" That question is strongly correlated with race, and yet is strongly arguable to be relevant. Model of car is hard to say that it's useful for predictive value, but home address (=which neighborhood did you grow up in, e.g., are you in Cabrini-Green or are you Gold Coast?) and annual income (poor people are more likely to commit crimes than rich people) definitely are going to be queried inputs. Particularly if the question is things like "what should I set bail at?"
- gravypod 10y ago> "Has the person previously been convicted of a crime?" That question is strongly correlated with race, and yet is strongly arguable to be relevant. Past behavior is a good predictor of future behavior in my experience. That's more than doubled when it's recurring behavior. > home address (=which neighborhood did you grow up in, e.g., are you in Cabrini-Green or are you Gold Coast?) and annual income (poor people are more likely to commit crimes than rich people) definitely are going to be queried inputs Why? > Particularly if the question is things like "what should I set bail at?" That's still the job of the judge and there is law at hand for how those values are calculated. The computers won't be involved with that (I'd assume) just sentencing based on previous case decisions.
- pdkl95 10y ago> Past behavior is a good predictor of future behavior in my experience. That question includes the past behavior of a lot more people than just the person with a criminal conviction. > recurring behavior Institutional racism has recurred a lot in the justice system.
- mattnewton 10y agoI agree they shouldn't be factors, and my example is a bit contrived, but many do include socioeconomic and location based factors, like this reply by the authors of a paper on one recidivism calculator (who argue here why socioeconomic factors are included) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5047350/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5047350/
- dilemma 10y ago>In the case you presented, the biases are in the dataset itself. The code that is going to handle the data is also biased in some direction. Its actions is value-based, coming from that of the creators of the algorithm. This is why all code must be open and publicly auditable if used in law enforcement, health care and insurance, etc.