4 ms·

I'm sure Vox would like if Facebook were obligated to promote approved opinions, but I'm not comfortable with any given group of people doing such sort of appro

I'm sure Vox would like if Facebook were obligated to promote approved opinions, but I'm not comfortable with any given group of people doing such sort of approval, so right now I consider an algorithmic approach natively superior.

jshevek 10y ago
I have to agree. It could be argued that websites like Vox harm our democracy more than Facebook.
privong 10y ago
> I'm not comfortable with any given group of people doing such sort of approval, so right now I consider an algorithmic approach natively superior. That is a point the article makes, though. The algorithmic approach is effectively equivalent to particular group of people doing the approval (or rejection). What group that is manifests itself via the way the algorithms are coded and the data that are fed into it. Edit: This is similar to the conclusion people are coming to regarding algorithms for court sentencing[0] and other areas where algorithms are becoming increasingly used. It's dangerous to conclude that an algorithm is better just by virtue of it being code. That code (and/or its training data) may be reinforcing existing stereotypes and biases. [0] https://www.engadget.com/2016/06/26/wisconsin-sentencing-algorithm-faces-court-battle/ https://www.engadget.com/2016/06/26/wisconsin-sentencing-alg...
- scarmig 10y ago
  It's not equivalent: it's designing a procedural system that produces certain outcomes. There's no reason to think those outcomes are inherently superior, but it's also unreasonable to think that it's equivalent to particular people doing the approval or rejection. It's similar to a hiring process: which is better, having a known procedure and rubric for choosing which candidates to hire, or just allow a manager to go with their gut and choose their nephew? There's a sense in which they are the same--at some level, designing a system must still take into account human values and biases. But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases.
  - zardo 10y ago
    >But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases. I don't see how that follows. The biases of an algorithm or beurocracy aren't inherently superior, they are just inherently repeatable.
    - scarmig 10y ago
      Being repeatable is valuable in itself: it makes the procedure testable and inherently more transparent. Yes, someone could come up with a hiring decision procedure that has as its first step "first, immediately reject all women and nonwhites." But that'd be subject to auditing and discovery in a way that a single grumpy old man finding arbitrary reasons to reject all women and nonwhites is not. You can iterate and improve on designed systems; you can't iterate and improve on human fallibility.
      - zardo 10y ago
        There's a basic assumption you have to make, anything we don't know how to quantify isn't worth considering. That's how we got standardized testing. We get fairness by treating everyone equally, without compassion, without empathy.
- yummyfajitas 10y ago
  The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is simply a theorem: it's mathematically impossible to be both well calibrated (a black and white person receiving the same risk score have the same probability of committing crime) and also racially balanced (similar levels of false positives), except in trivial and unrealistic cases. https://arxiv.org/pdf/1609.05807v1.pdf https://arxiv.org/pdf/1609.05807v1.pdf See also this WaPo article explaining in simpler terms: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... But the thing is, this theorem applies to any decision process - human or algorithm. You can't escape mathematical impossibility results just by using humans (e.g. NP Complete problems are still hard even if done by humans). Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic.
  - urish 10y ago
    My only disagreement is with: >Human processes just add additional bias Bias with respect to what? As you say, there is already bias baked into the data collection and the algorithmic choices. The bias that human editors introduce is different, but not necessarily larger, however you even measure it. There are also myriad human choices behind the choice and deployment details of the algorithm. An important plus for human editors is greater interpretability and greater transparency regarding the biases the system ends up showing.
    - yummyfajitas 10y ago
      It's been reproduced across many experiments that humans will add bias that harms accuracy when making decisions. I.e., if x[6] represents race, humans will systematically wrongly weight x[6]. Machines simply don't do this. As you say, there is already bias baked into the data collection and the algorithmic choices. That's not what I said. What I said is that you can't have collective equality (e.g. same rate of false positives, lack of disparate impact) and also accuracy (getting the right answer) except in trivial/unrealistic cases. Human editors are fundamentally less interpretable and transparent than machines. You can easily interrogate machines and test for bias; how do you do that to humans? Or, to take a historical example, why did colleges switch from algorithms to humans when the supreme court said that transparent racial bias is forbidden?
superbatfish 10y ago
>I'm not comfortable with any given group of people doing such sort of approval Historically, that's exactly what newspaper editors did. The algorithmic approach has the potential to offer greater transparency, but not if the algorithm isn't published. Furthermore, now that Facebook is in a near-monopolistic position when it comes to picking winners and losers in what we read, acknowledging that fact publicly would be the bare-minimum first step they could take towards accountability. Then we can have a more open discussion about how news stories ought to be judged and propagated (or not).
- yummyfajitas 10y ago
  The algorithmic approach has the potential to offer greater transparency, but not if the algorithm isn't published. This isn't really true. There's actually a mathematical discipline devoted to treating statistical algorithms as black boxes, and assessing their accuracy/generalizability based solely on test data/broad intrinsic properties of the algorithm (e.g. continuity parameters). It's called "machine learning". From what I can tell, the fundamental distinction between machine learning and statistics is that machine learning provides mathematical guarantees based solely on black box testing of algorithms and very broad properties of the distribution/algo. In contrast, statistics cares about the actual underlying process.
  - superbatfish 10y ago
    Great points. It's true that whatever algorithm they end up with will definitely use ML. And it's also true that "interpretability" is one of the great challenges for ML in the coming years. But interpretability of a model isn't a binary attribute. Some techniques offer at least some intuition about why they chose their results. But a human editor, on the other hand, is much less transparent -- basically, the best justification they can offer is "trust me, I know what I'm doing".

jshevek 10y ago

I have to agree. It could be argued that websites like Vox harm our democracy more than Facebook.

privong 10y ago

> I'm not comfortable with any given group of people doing such sort of approval, so right now I consider an algorithmic approach natively superior. That is a point the article makes, though. The algorithmic approach is effectively equivalent to particular group of people doing the approval (or rejection). What group that is manifests itself via the way the algorithms are coded and the data that are fed into it. Edit: This is similar to the conclusion people are coming to regarding algorithms for court sentencing[0] and other areas where algorithms are becoming increasingly used. It's dangerous to conclude that an algorithm is better just by virtue of it being code. That code (and/or its training data) may be reinforcing existing stereotypes and biases. [0] https://www.engadget.com/2016/06/26/wisconsin-sentencing-algorithm-faces-court-battle/ https://www.engadget.com/2016/06/26/wisconsin-sentencing-alg...

scarmig 10y ago
It's not equivalent: it's designing a procedural system that produces certain outcomes. There's no reason to think those outcomes are inherently superior, but it's also unreasonable to think that it's equivalent to particular people doing the approval or rejection. It's similar to a hiring process: which is better, having a known procedure and rubric for choosing which candidates to hire, or just allow a manager to go with their gut and choose their nephew? There's a sense in which they are the same--at some level, designing a system must still take into account human values and biases. But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases.
- zardo 10y ago
  >But that's still a huge step forward from having people, even smart people, make gut decisions just based on what's in their head, even if that's still a decision driven by human values and biases. I don't see how that follows. The biases of an algorithm or beurocracy aren't inherently superior, they are just inherently repeatable.
  - scarmig 10y ago
    Being repeatable is valuable in itself: it makes the procedure testable and inherently more transparent. Yes, someone could come up with a hiring decision procedure that has as its first step "first, immediately reject all women and nonwhites." But that'd be subject to auditing and discovery in a way that a single grumpy old man finding arbitrary reasons to reject all women and nonwhites is not. You can iterate and improve on designed systems; you can't iterate and improve on human fallibility.
    - zardo 10y ago
      There's a basic assumption you have to make, anything we don't know how to quantify isn't worth considering. That's how we got standardized testing. We get fairness by treating everyone equally, without compassion, without empathy.
yummyfajitas 10y ago
The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is simply a theorem: it's mathematically impossible to be both well calibrated (a black and white person receiving the same risk score have the same probability of committing crime) and also racially balanced (similar levels of false positives), except in trivial and unrealistic cases. https://arxiv.org/pdf/1609.05807v1.pdf https://arxiv.org/pdf/1609.05807v1.pdf See also this WaPo article explaining in simpler terms: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/ https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1... But the thing is, this theorem applies to any decision process - human or algorithm. You can't escape mathematical impossibility results just by using humans (e.g. NP Complete problems are still hard even if done by humans). Human processes just add additional bias, e.g. the editorial slant that Vox wants Facebook to add, or the (alleged) bias that Facebook's human editors added before Facebook went all algorithmic.
- urish 10y ago
  My only disagreement is with: >Human processes just add additional bias Bias with respect to what? As you say, there is already bias baked into the data collection and the algorithmic choices. The bias that human editors introduce is different, but not necessarily larger, however you even measure it. There are also myriad human choices behind the choice and deployment details of the algorithm. An important plus for human editors is greater interpretability and greater transparency regarding the biases the system ends up showing.
  - yummyfajitas 10y ago
    It's been reproduced across many experiments that humans will add bias that harms accuracy when making decisions. I.e., if x[6] represents race, humans will systematically wrongly weight x[6]. Machines simply don't do this. As you say, there is already bias baked into the data collection and the algorithmic choices. That's not what I said. What I said is that you can't have collective equality (e.g. same rate of false positives, lack of disparate impact) and also accuracy (getting the right answer) except in trivial/unrealistic cases. Human editors are fundamentally less interpretable and transparent than machines. You can easily interrogate machines and test for bias; how do you do that to humans? Or, to take a historical example, why did colleges switch from algorithms to humans when the supreme court said that transparent racial bias is forbidden?

superbatfish 10y ago

>I'm not comfortable with any given group of people doing such sort of approval Historically, that's exactly what newspaper editors did. The algorithmic approach has the potential to offer greater transparency, but not if the algorithm isn't published. Furthermore, now that Facebook is in a near-monopolistic position when it comes to picking winners and losers in what we read, acknowledging that fact publicly would be the bare-minimum first step they could take towards accountability. Then we can have a more open discussion about how news stories ought to be judged and propagated (or not).

yummyfajitas 10y ago
The algorithmic approach has the potential to offer greater transparency, but not if the algorithm isn't published. This isn't really true. There's actually a mathematical discipline devoted to treating statistical algorithms as black boxes, and assessing their accuracy/generalizability based solely on test data/broad intrinsic properties of the algorithm (e.g. continuity parameters). It's called "machine learning". From what I can tell, the fundamental distinction between machine learning and statistics is that machine learning provides mathematical guarantees based solely on black box testing of algorithms and very broad properties of the distribution/algo. In contrast, statistics cares about the actual underlying process.
- superbatfish 10y ago
  Great points. It's true that whatever algorithm they end up with will definitely use ML. And it's also true that "interpretability" is one of the great challenges for ML in the coming years. But interpretability of a model isn't a binary attribute. Some techniques offer at least some intuition about why they chose their results. But a human editor, on the other hand, is much less transparent -- basically, the best justification they can offer is "trust me, I know what I'm doing".