3 ms·

The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is

The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is simply a theorem: it's mathematically impossible to be both well calibrated (a black and white person receiving the same risk score have the same probability of committing crime) and also racially balanced (similar levels of false positives), except in trivial and unrealistic cases. https://arxiv.org/pdf/1609.05807v1.pdf https://arxiv.org/pdf/1609.05807v1.pdf See also this WaPo article explaining in simpler terms: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... But the thing is, this theorem applies to any decision process - human or algorithm. You can't escape mathematical impossibility results just by using humans (e.g. NP Complete problems are still hard even if done by humans). Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic.

urish 10y ago
My only disagreement is with: >Human processes just add additional bias Bias with respect to what? As you say, there is already bias baked into the data collection and the algorithmic choices. The bias that human editors introduce is different, but not necessarily larger, however you even measure it. There are also myriad human choices behind the choice and deployment details of the algorithm. An important plus for human editors is greater interpretability and greater transparency regarding the biases the system ends up showing.
- yummyfajitas 10y ago
  It's been reproduced across many experiments that humans will add bias that harms accuracy when making decisions. I.e., if x[6] represents race, humans will systematically wrongly weight x[6]. Machines simply don't do this. As you say, there is already bias baked into the data collection and the algorithmic choices. That's not what I said. What I said is that you can't have collective equality (e.g. same rate of false positives, lack of disparate impact) and also accuracy (getting the right answer) except in trivial/unrealistic cases. Human editors are fundamentally less interpretable and transparent than machines. You can easily interrogate machines and test for bias; how do you do that to humans? Or, to take a historical example, why did colleges switch from algorithms to humans when the supreme court said that transparent racial bias is forbidden?
privong 10y ago
> The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. Okay, that may have not been the best example to reference. > Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic. But how you can craft an algorithm that takes human processes (clickbait, investigative journalism, breaking news, photos of cats) and provides an output free of biases? Given the fluid nature of the input biases, is it even possible to craft an algorithm that accounts for those biases without adding in (or substituting) other biases? What is the "objective standard" by which you measure and correct the input biases? Just because there's an algorithm doesn't mean the output is bias-free.

urish 10y ago

My only disagreement is with: >Human processes just add additional bias Bias with respect to what? As you say, there is already bias baked into the data collection and the algorithmic choices. The bias that human editors introduce is different, but not necessarily larger, however you even measure it. There are also myriad human choices behind the choice and deployment details of the algorithm. An important plus for human editors is greater interpretability and greater transparency regarding the biases the system ends up showing.

yummyfajitas 10y ago
It's been reproduced across many experiments that humans will add bias that harms accuracy when making decisions. I.e., if x[6] represents race, humans will systematically wrongly weight x[6]. Machines simply don't do this. As you say, there is already bias baked into the data collection and the algorithmic choices. That's not what I said. What I said is that you can't have collective equality (e.g. same rate of false positives, lack of disparate impact) and also accuracy (getting the right answer) except in trivial/unrealistic cases. Human editors are fundamentally less interpretable and transparent than machines. You can easily interrogate machines and test for bias; how do you do that to humans? Or, to take a historical example, why did colleges switch from algorithms to humans when the supreme court said that transparent racial bias is forbidden?

privong 10y ago

> The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. Okay, that may have not been the best example to reference. > Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic. But how you can craft an algorithm that takes human processes (clickbait, investigative journalism, breaking news, photos of cats) and provides an output free of biases? Given the fluid nature of the input biases, is it even possible to craft an algorithm that accounts for those biases without adding in (or substituting) other biases? What is the "objective standard" by which you measure and correct the input biases? Just because there's an algorithm doesn't mean the output is bias-free.