4 ms·

Machine Learning Algorithms are severely racist towards defendants. We know, for example, that AI algorithms mistake Black people for primates... but they are

Machine Learning Algorithms are severely racist towards defendants. We know, for example, that AI algorithms mistake Black people for primates... but they are no more racist than the criminal justice system at large. And they have a better chance of getting fixed. https://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white-guy-problem.html https://www.nytimes.com/2016/06/26/opinion/sunday/artificial...

drharby 8y ago
Wow citation needed, and a nyt opinion piece is not research proving your claim. "Algorithms are racist" ha!
- tantalor 8y ago
  Here's a few, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... https://www.technologyreview.com/s/607955/inspecting-algorithms-for-bias/ https://www.technologyreview.com/s/607955/inspecting-algorit... http://fortune.com/longform/ai-bias-problem/ http://fortune.com/longform/ai-bias-problem/ https://www.pcmag.com/article/361661/artificial-intelligence-has-a-bias-problem-and-its-our-fau https://www.pcmag.com/article/361661/artificial-intelligence... Ethics and Bias in Machine Learning: A Technical Study of What Makes Us “Good” https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1042&context=jj_etds https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1...
- drharby 8y ago
  Man - to hell with this forum, I ask for evidence on a claim that other posters have pointed out is a misapplication of other research 'webscam algos != cash bail algos' And I get downvoted!? This is tech news, something the general public can get easily manipulated into...and for demanding a degree of reasonable discourse and reference, i get downvoted? Bleh, this place is turning into a place i cant speak relatively freely in.
  - Stratoscope 8y ago
    I doubt it was your request for evidence that got you downvoted, it was the way you phrased both the request and the follow-up comment. Try a more polite tone and you may get better results.
moosey 8y ago
I find it more disturbing that AI algorithms don't think that non-black people are primates, as I can assure you they are.
pc86 8y ago
Why is it that the vast majority of the time I see "We know[, for example,]" at the start of a sentence, the text following is one or more of: 1. A blatant lie 2. A willful misinterpretation of new research 3. A gross misunderstanding of research outside one's area of expertise 4. A misapplication of one thing for another, e.g. conflating webcam algorithms and all their issues with criminal justice algorithms
SargeZT 8y ago
New Jersey's system has proven to be remarkably fair. It turns out when you design the system specifically not to be racist, it isn't racist. Also, I'm not sure how misidentification of pictures relates to the topic at hand.
- cwmma 8y ago
  > It turns out when you design the system specifically not to be racist, it isn't racist. This is key, New Jersey went out of it's wait to avoid using anything that could be a proxy for race like zip codes. They also used big data to come up with the algorithm, the algorithm itself[1] is pretty strait forward and is not a black box. The fear in California is that some counties will use black box ML systems that WILL have bias even if it's not intentional, just like how biases in the training data of image recognition tend to misidentify pictures of black people more then white people. 1. https://www.arnoldfoundation.org/wp-content/uploads/PSA-Risk-Factors-and-Formula.pdf https://www.arnoldfoundation.org/wp-content/uploads/PSA-Risk...
Anderkent 8y ago
Bond is severely racist too, if you take those criteria. Black people get assigned higher bond, and often have a harder time getting it together. As such there are many people in jail that are in no way a risk but are held because they can't afford bond.
- panic 8y ago
  From this perspective, bond-based pretrial detention is actually better -- communities can organize bail funds to reduce its impact. You can't really fight algorithmic detention in the same way.
- rectang 8y ago
  Our awareness that machine learning algorithms have the potential to make racist decisions gives us something to watch out for as these systems roll out. Hopefully addressing any flaws will be a matter of ordinary debugging. Regardless, the cash bail system is terrible and I'm anticipating big improvements as we move away from it.

drharby 8y ago

Wow citation needed, and a nyt opinion piece is not research proving your claim. "Algorithms are racist" ha!

tantalor 8y ago
Here's a few, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... https://www.technologyreview.com/s/607955/inspecting-algorithms-for-bias/ https://www.technologyreview.com/s/607955/inspecting-algorit... http://fortune.com/longform/ai-bias-problem/ http://fortune.com/longform/ai-bias-problem/ https://www.pcmag.com/article/361661/artificial-intelligence-has-a-bias-problem-and-its-our-fau https://www.pcmag.com/article/361661/artificial-intelligence... Ethics and Bias in Machine Learning: A Technical Study of What Makes Us “Good” https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1042&context=jj_etds https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1...
drharby 8y ago
Man - to hell with this forum, I ask for evidence on a claim that other posters have pointed out is a misapplication of other research 'webscam algos != cash bail algos' And I get downvoted!? This is tech news, something the general public can get easily manipulated into...and for demanding a degree of reasonable discourse and reference, i get downvoted? Bleh, this place is turning into a place i cant speak relatively freely in.
- Stratoscope 8y ago
  I doubt it was your request for evidence that got you downvoted, it was the way you phrased both the request and the follow-up comment. Try a more polite tone and you may get better results.

moosey 8y ago

I find it more disturbing that AI algorithms don't think that non-black people are primates, as I can assure you they are.

pc86 8y ago

Why is it that the vast majority of the time I see "We know[, for example,]" at the start of a sentence, the text following is one or more of: 1. A blatant lie 2. A willful misinterpretation of new research 3. A gross misunderstanding of research outside one's area of expertise 4. A misapplication of one thing for another, e.g. conflating webcam algorithms and all their issues with criminal justice algorithms

SargeZT 8y ago

New Jersey's system has proven to be remarkably fair. It turns out when you design the system specifically not to be racist, it isn't racist. Also, I'm not sure how misidentification of pictures relates to the topic at hand.

cwmma 8y ago
> It turns out when you design the system specifically not to be racist, it isn't racist. This is key, New Jersey went out of it's wait to avoid using anything that could be a proxy for race like zip codes. They also used big data to come up with the algorithm, the algorithm itself[1] is pretty strait forward and is not a black box. The fear in California is that some counties will use black box ML systems that WILL have bias even if it's not intentional, just like how biases in the training data of image recognition tend to misidentify pictures of black people more then white people. 1. https://www.arnoldfoundation.org/wp-content/uploads/PSA-Risk-Factors-and-Formula.pdf https://www.arnoldfoundation.org/wp-content/uploads/PSA-Risk...

Anderkent 8y ago

Bond is severely racist too, if you take those criteria. Black people get assigned higher bond, and often have a harder time getting it together. As such there are many people in jail that are in no way a risk but are held because they can't afford bond.

panic 8y ago
From this perspective, bond-based pretrial detention is actually better -- communities can organize bail funds to reduce its impact. You can't really fight algorithmic detention in the same way.
rectang 8y ago
Our awareness that machine learning algorithms have the potential to make racist decisions gives us something to watch out for as these systems roll out. Hopefully addressing any flaws will be a matter of ordinary debugging. Regardless, the cash bail system is terrible and I'm anticipating big improvements as we move away from it.