4 ms·

> I'm not comfortable with any given group of people doing such sort of approval, so right now I consider an algorithmic approach natively superior. That is a

by privong 10y ago

> I'm not comfortable with any given group of people doing such sort of approval, so right now I consider an algorithmic approach natively superior. That is a point the article makes, though. The algorithmic approach is effectively equivalent to particular group of people doing the approval (or rejection). What group that is manifests itself via the way the algorithms are coded and the data that are fed into it. Edit: This is similar to the conclusion people are coming to regarding algorithms for court sentencing[0] and other areas where algorithms are becoming increasingly used. It's dangerous to conclude that an algorithm is better just by virtue of it being code. That code (and/or its training data) may be reinforcing existing stereotypes and biases. [0] https://www.engadget.com/2016/06/26/wisconsin-sentencing-algorithm-faces-court-battle/ https://www.engadget.com/2016/06/26/wisconsin-sentencing-alg...

scarmig 10y ago
It's not equivalent: it's designing a procedural system that produces certain outcomes. There's no reason to think those outcomes are inherently superior, but it's also unreasonable to think that it's equivalent to particular people doing the approval or rejection. It's similar to a hiring process: which is better, having a known procedure and rubric for choosing which candidates to hire, or just allow a manager to go with their gut and choose their nephew? There's a sense in which they are the same--at some level, designing a system must still take into account human values and biases. But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases.
- zardo 10y ago
  >But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases. I don't see how that follows. The biases of an algorithm or beurocracy aren't inherently superior, they are just inherently repeatable.
  - scarmig 10y ago
    Being repeatable is valuable in itself: it makes the procedure testable and inherently more transparent. Yes, someone could come up with a hiring decision procedure that has as its first step "first, immediately reject all women and nonwhites." But that'd be subject to auditing and discovery in a way that a single grumpy old man finding arbitrary reasons to reject all women and nonwhites is not. You can iterate and improve on designed systems; you can't iterate and improve on human fallibility.
    - zardo 10y ago
      There's a basic assumption you have to make, anything we don't know how to quantify isn't worth considering. That's how we got standardized testing. We get fairness by treating everyone equally, without compassion, without empathy.
yummyfajitas 10y ago
The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is simply a theorem: it's mathematically impossible to be both well calibrated (a black and white person receiving the same risk score have the same probability of committing crime) and also racially balanced (similar levels of false positives), except in trivial and unrealistic cases. https://arxiv.org/pdf/1609.05807v1.pdf https://arxiv.org/pdf/1609.05807v1.pdf See also this WaPo article explaining in simpler terms: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... But the thing is, this theorem applies to any decision process - human or algorithm. You can't escape mathematical impossibility results just by using humans (e.g. NP Complete problems are still hard even if done by humans). Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic.
- urish 10y ago
  My only disagreement is with: >Human processes just add additional bias Bias with respect to what? As you say, there is already bias baked into the data collection and the algorithmic choices. The bias that human editors introduce is different, but not necessarily larger, however you even measure it. There are also myriad human choices behind the choice and deployment details of the algorithm. An important plus for human editors is greater interpretability and greater transparency regarding the biases the system ends up showing.
  - yummyfajitas 10y ago
    It's been reproduced across many experiments that humans will add bias that harms accuracy when making decisions. I.e., if x[6] represents race, humans will systematically wrongly weight x[6]. Machines simply don't do this. As you say, there is already bias baked into the data collection and the algorithmic choices. That's not what I said. What I said is that you can't have collective equality (e.g. same rate of false positives, lack of disparate impact) and also accuracy (getting the right answer) except in trivial/unrealistic cases. Human editors are fundamentally less interpretable and transparent than machines. You can easily interrogate machines and test for bias; how do you do that to humans? Or, to take a historical example, why did colleges switch from algorithms to humans when the supreme court said that transparent racial bias is forbidden?

by privong 10y ago

scarmig 10y ago

It's not equivalent: it's designing a procedural system that produces certain outcomes. There's no reason to think those outcomes are inherently superior, but it's also unreasonable to think that it's equivalent to particular people doing the approval or rejection. It's similar to a hiring process: which is better, having a known procedure and rubric for choosing which candidates to hire, or just allow a manager to go with their gut and choose their nephew? There's a sense in which they are the same--at some level, designing a system must still take into account human values and biases. But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases.

zardo 10y ago
>But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases. I don't see how that follows. The biases of an algorithm or beurocracy aren't inherently superior, they are just inherently repeatable.
- scarmig 10y ago
  Being repeatable is valuable in itself: it makes the procedure testable and inherently more transparent. Yes, someone could come up with a hiring decision procedure that has as its first step "first, immediately reject all women and nonwhites." But that'd be subject to auditing and discovery in a way that a single grumpy old man finding arbitrary reasons to reject all women and nonwhites is not. You can iterate and improve on designed systems; you can't iterate and improve on human fallibility.
  - zardo 10y ago
    There's a basic assumption you have to make, anything we don't know how to quantify isn't worth considering. That's how we got standardized testing. We get fairness by treating everyone equally, without compassion, without empathy.

yummyfajitas 10y ago

The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is simply a theorem: it's mathematically impossible to be both well calibrated (a black and white person receiving the same risk score have the same probability of committing crime) and also racially balanced (similar levels of false positives), except in trivial and unrealistic cases. https://arxiv.org/pdf/1609.05807v1.pdf https://arxiv.org/pdf/1609.05807v1.pdf See also this WaPo article explaining in simpler terms: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... But the thing is, this theorem applies to any decision process - human or algorithm. You can't escape mathematical impossibility results just by using humans (e.g. NP Complete problems are still hard even if done by humans). Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic.

urish 10y ago
My only disagreement is with: >Human processes just add additional bias Bias with respect to what? As you say, there is already bias baked into the data collection and the algorithmic choices. The bias that human editors introduce is different, but not necessarily larger, however you even measure it. There are also myriad human choices behind the choice and deployment details of the algorithm. An important plus for human editors is greater interpretability and greater transparency regarding the biases the system ends up showing.
- yummyfajitas 10y ago
  It's been reproduced across many experiments that humans will add bias that harms accuracy when making decisions. I.e., if x[6] represents race, humans will systematically wrongly weight x[6]. Machines simply don't do this. As you say, there is already bias baked into the data collection and the algorithmic choices. That's not what I said. What I said is that you can't have collective equality (e.g. same rate of false positives, lack of disparate impact) and also accuracy (getting the right answer) except in trivial/unrealistic cases. Human editors are fundamentally less interpretable and transparent than machines. You can easily interrogate machines and test for bias; how do you do that to humans? Or, to take a historical example, why did colleges switch from algorithms to humans when the supreme court said that transparent racial bias is forbidden?