5 ms·

You're going to have to break the Turing Test before you get good automated content curation. There will always be error. People are worried about bias. I'm sk

by mjfl 10y ago

You're going to have to break the Turing Test before you get good automated content curation. There will always be error. People are worried about bias. I'm skeptical that humans have the ability to be politically unbiased on an individual level. Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide. So let's stop pretending to be unbiased. Information aggregators should have their members clearly spell out their positions on popular issues in their bios (an openness that should be a part of the responsibility of being an information aggregator) and there should be "block designed" hiring around these issues to maximize political diversity and reduce bias in aggregate. Otherwise network effects are just going to cluster people with similar opinions around each other. Just a thought.

jcoffland 10y ago
How do you prove that you believe something?
- mjfl 10y ago
  I think it's extremely hard to reverse engineer passionate defense of an issue by a true believer. For example, if you were body-swapped with the late justice Antonin Scalia, I find it highly unlikely that you would be able to argue as vigorously for the conservative cause if you had lived your life as a liberal Silicon Valley tech geek (they'd also see right through you because you'd be a zombie and clearly not the true Scalia). So that's one thing. Perhaps in some extreme pathological case you could have a person live their entire lives faking the opposite political ideology. But that just seems like a waste of energy... And if you can fake being for an issue convincingly enough, doesn't that have the same effect as being for the issue anyways? Like, people would catch on if you wrote about being a Bernie Sanders supporter in your bio, but wrote/submitted all articles about how he doesn't have enough foreign policy experience. The only way to fake being a supporter of something is to.. consistently make solid arguments in support of it.
- pessimizer 10y ago
  By not having anything to sell.
- tdaltonc 10y ago
  It's hard to fake an IAT. https://implicit.harvard.edu/implicit/education.html https://implicit.harvard.edu/implicit/education.html
  - dlss 10y ago
    In my experience it's relatively easy if you essentially treat an IAT test as a method acting challenge. According to the literature it's relatively easy for study participants when given a couple tries to get it "right": http://psycnet.apa.org/psycinfo/2004-17684-002 http://psycnet.apa.org/psycinfo/2004-17684-002
  - jcoffland 10y ago
    Is it? I believe it's rather easy actually.
hiou 10y ago
Because optimizing for engagement is the modern equivalent of pandering?
- derefr 10y ago
  It'd be interesting to optimize for other factors. I'd like to see a news source that tries to optimize its presentation to incite the least possible controversy/argument while keeping engagement high. I'd guess this would be done by thoroughly explaining "both sides" of an issue (not in "equal times" terms, but in "all that can be said, has been said" terms)—but who knows, maybe there's a clever trick to doing something like this that such an algorithm would force into existence, much like PageRank forced the SEO industry into existence.
- mjfl 10y ago
  My comment assumed we were optimizing to minimize bias - which people do care about - and gave an approach to solve that. Optimizing for engagement is fine, and profitable. You just can't claim to be unbiased.
derefr 10y ago
I have a hypothesis that a lot of articles "bury the lede" of their political affiliation (both from ML algorithms and from people) because journalists believe that they can "save the world" by tricking people on one side of an issue into reading arguments on the other side of that issue. Whereas—they believe—if such articles are clearly marked, people on the "other side" will just avoid them and stay happily in their filter-bubbles.
majormajor 10y ago
> take abortion, you are either for choice, or you believe it is infanticide Well, no, even on abortion there is usually a position somewhere in the middle is where a lot of people end up (and you left out part of the spectrum, where even contraceptives are believed to be morally wrong), and usually the law has ended up somewhere in the middle, too. We confuse "loudest" views with "most common" a lot these days, so I do agree that promoting the most extreme views possible to drive "engagement" (a tactic figured out by people looking for votes long before the internet was even around to introduce engagement->clicks->dollars) is a bad behavior for aggregators, but that's a hard argument to make for a business as long as we hold the current view of short-term "fiduciary duty" ruling all.
- humanrebar 10y ago
  Some "middle positions" on abortion tend to be poorly thought out from a deontological perspective. Allowing abortion in cases of rape makes zero sense if abortion is murder, for example. Help should surely be given to rape victims, but giving it in the form of infanticide?
  - dhimes 10y ago
    I understand your point but I don't think we should continue on the particular topic of abortion in this thread.
- Lawtonfogle 10y ago
  When considering all abortions there are many positions which limit different subsets as being wrong, but for any given situation one must choose between abortion allowed or abortion banned (and technically the choice of abortion required but that almost never pops up). So while one might be against partial birth abortion but for second trimester abortions, they still have to look at a given situation and say yay or nay.
mnarayan01 10y ago
take abortion, you are either for choice, or you believe it is infanticide The mere existence of the term "infanticide" suggests that the spectrum of opinions is less binary than you're making it out to be.
JumpCrisscross 10y ago
> take abortion, you are either for choice, or you believe it is infanticide I don't. It's fundamentally a question about when sex cells become a person, and if rights accumulate sequentially or instantaneously. There are lots of lines one can reasonably draw, e.g. independent viability (downside: disenfranchises premature babies), consciousness (no way to measure it), neural activity (how much?), heartbeat (why not if the tongue's wiggling?), conception, birth, et cetera. Breaking down problems is one way to help dis-engage tribal instincts.
- mjfl 10y ago
  It is my perception that people with the middle views you describe would still be considered "pro-choice" and would largely vote "pro-choice" but perhaps I am wrong. If those people were split 50/50 in the way that they vote I would concede that there is a middle ground in that issue.
  - deleted 10y ago
    [deleted]
tuna-piano 10y ago
Should they also seek to hire neo-nazis to get the nazi viewpoint as well?
- dlss 10y ago
  Downvoted you because I don't quite see how this connects with GP other than a by being a Godwin laden form of "I'm not going to explain why, but trust me you're wrong" Worth a read IMO: http://www.paulgraham.com/disagree.html http://www.paulgraham.com/disagree.html
- ommunist 10y ago
  by the way, very interesting, where can I get that viewpoint?
- venomsnake 10y ago
  Yes of course. After all we strife for diversity, accrptance and tolerance. Unless we actually strife for cultural hegemony snd eradicating the wrong thoughts from the people that are currently losing the discourse.
leepowers 10y ago
> You're going to have to break the Turing Test before you get good automated content curation. There will always be error. Yes, but even a competent AI curation system is going to make mistakes. The only relevant question is "what is an acceptable rate of error?". (Hint: 0% is not an acceptable answer.) We may be able to get to an acceptable rate (or may already be there) with or without AI-complete. > People are worried about bias ... Information aggregators should have their members clearly spell out their positions on popular issues This isn't a problem that can be solved with meta-data. An aggregator could write a beautiful, open-minded mission statement and still write trash. Hell, Google's unofficial mantra is "don't do evil". Well, ok then, there's nothing to worry about, Google has clearly spelled out their position, so there's no need to analyze or criticize their policies. A policy statement or any other meta-data is not going to solve this. As you correctly noted an AI that passes the TT may be one way to work around this. But I'm skeptical. Any such AI would be at least as complicated as human intelligence, including all the foibles and errors that entails. An AI without our cognitive hang-ups would be smarter and more capable than us. Which would not increase my trust, and possibly decrease it, as the AI would no doubt have a bias to curate stories in it's own favor, or that put it (and AI in general) in the best light. Bias is an unavoidable component of any curation.
- PhasmaFelis 10y ago
  > The only relevant question is "what is an acceptable rate of error?". (Hint: 0% is not an acceptable answer.) Certainly it is. That just means you're admitting that the problem cannot be satisfactorily solved by current AI and must be left to humans for now.
  - Sacho 10y ago
    I doubt the error rate was 0% with human curators. In fact, the article itself does not compare the error rate of the human curators to algorithm, just examines that it was above zero after the humans were removed. I think an acceptable statement to make would be that the human curators probably reduced the number of "fake stories", but that's not why they were removed. It's a different judgement call whether the trending feed is "better" or "worse" in some holistic sense. Furthermore, there seems to be an implicit assumption that Facebook's Trending stories should not contain "fake stories", but is that the intended goal of the service? I actually have no idea, since I've never used it.
unityByFreedom 10y ago
> there should be "block designed" hiring around these issues to maximize political diversity and reduce bias in aggregate How do we deal with those who would push treasonous information? If WikiLeaks publishes correspondence between, say, USA and China, that is private because China wanted it that way, and that hurts American allies, should this be allowed to trend on an American-based website? Has WikiLeaks contributed to the right wing propaganda we see being pushed online? Should some of this be punishable?
- dlss 10y ago
  These are very good questions, and have been debated at extensively (as you might imagine for such important questions) by humanity's legal, ethical, and philosophical scholars. Here's the current state of things with references as to why: https://en.wikipedia.org/wiki/Freedom_of_speech_in_the_United_States https://en.wikipedia.org/wiki/Freedom_of_speech_in_the_Unite... although I have some emotional disagreement with those policies at times, I can't say I'm well read enough on the topic to really have a strong disagreement personally.
  - unityByFreedom 10y ago
    Thanks. I mention it because, as a tech enthusiast, I was once very much in the camp of "we should know everything our government does" Now, I understand that "we can't know everything, because some other governments won't tell us their true feelings if we don't keep those conversations private." Perhaps one day some Chinese or Russian hackers will level the playing field. In the mean time, I wonder if more tech folks can understand that our government might actually want to prosecute Snowden, Assange etc. for legitimate threats to our national security.
FullMtlAlcoholc 10y ago
>Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide. So let's stop pretending to be unbiased. You're falling into the trap of American politics by making every issue a binary choice. For instance, I find the abortion debate to be such a non-issue and if someone asks me what my stance is, I respond by saying I haven't done enough research or soul-searching to form a coherent, rational position on this issue. And I probably will not ever get around to doing this research. I'm ok not having a position on certain issues. I wish this was more acceptable. Instead, the US political duopoly and it's first to the post voting encouraged big tent parties who more or less have to agree on most issues. I'm going to paraphrase Rumsfeld's brilliant turn of phrase here, there are known unknowns, and there's power in accepting that.
- lucio 10y ago
  You should note that you have the luxury to even consider the issue of abortion because you've already been born.
  - snovv_crash 10y ago
    Wow, that's a pretty dim view of humanity you have there. They were also conceived. Should contraceptives be illegal too? Just because there is a selection bias doesn't mean that the bias results in specifically positive or negative outcome. I could argue that because you were born you are against abortion because you were born too, and my argument would be just as valid. Of course I'm not making that argument, because it is just as silly as the one you are making.
  - leohutson 10y ago
    They are also lucky their parents met, so your point is?
    - hueving 10y ago
      Tends to be a less inflammatory 'pro life' way of saying, "it's easy to be undecided when the decision isn't about you being murdered." Or something provocative along those lines.
artimaeis 10y ago
> Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide. Hogwash. First, there's the spectrum of finding where human life begins. There's many stances here from at the point of conception to post-birth, and everywhere between. Then there's the factor of a woman's right to her body and life. Given the first factor - we have a whole new complex spectrum here where we weigh in the life of the infant vs the life of the mother. To further complicate this, there's the woman's right to privacy. This was the basis by which Roe v Wade was decided in the first place. So there's at least 3 spectrum on which any citizen can find themselves in the middle of in that issue. I agree that humans probably don't have the ability to be politically unbiased. I'm just not sure that bias matters that much. But to pretend that there are issues without middle ground is to fall into the trap that so many groups face these days - that divisive and non-empathetic stance will tear at us and divide us further. There's simply no place for it.
Bartweiss 10y ago
The Turing Test thing was my first reaction also. These are sites consciously imitating the stylistic markers of real news, where the only signal for falsification is the content of the story. They regularly fool real people - sometimes because The Onion's sarcasm slips past someone, sometimes because the piece is 100% plausible, yet false. Facebook could save itself some predictable pain by blacklisting The Onion and SportsPickle and other high-profile fake news sites, but they aren't going to be able to crack this problem with algorithms (or if they do, those algorithms will have far better uses).
mrob 10y ago
>Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide Those two options are not mutually exclusive. All healthy humans have an instinctive aversion to murder, and murder is universally considered morally wrong, but they do not have such an aversion to infanticide. In many cultures infanticide has been considered acceptable, and even in modern American people aren't going to ask too many questions if a new mother claims it was an accident. A new-born baby does not speak, it does not use tools, it fails the mirror test. By any objective measure it's less of a person than many animals we kill without a second thought. There's a strong argument that the mother should be allowed to make that decision, possibly even up to a short time (eg. 24 hours) post-birth.

by mjfl 10y ago

jcoffland 10y ago

How do you prove that you believe something?

mjfl 10y ago
I think it's extremely hard to reverse engineer passionate defense of an issue by a true believer. For example, if you were body-swapped with the late justice Antonin Scalia, I find it highly unlikely that you would be able to argue as vigorously for the conservative cause if you had lived your life as a liberal Silicon Valley tech geek (they'd also see right through you because you'd be a zombie and clearly not the true Scalia). So that's one thing. Perhaps in some extreme pathological case you could have a person live their entire lives faking the opposite political ideology. But that just seems like a waste of energy... And if you can fake being for an issue convincingly enough, doesn't that have the same effect as being for the issue anyways? Like, people would catch on if you wrote about being a Bernie Sanders supporter in your bio, but wrote/submitted all articles about how he doesn't have enough foreign policy experience. The only way to fake being a supporter of something is to.. consistently make solid arguments in support of it.
pessimizer 10y ago
By not having anything to sell.
tdaltonc 10y ago
It's hard to fake an IAT. https://implicit.harvard.edu/implicit/education.html https://implicit.harvard.edu/implicit/education.html
- dlss 10y ago
  In my experience it's relatively easy if you essentially treat an IAT test as a method acting challenge. According to the literature it's relatively easy for study participants when given a couple tries to get it "right": http://psycnet.apa.org/psycinfo/2004-17684-002 http://psycnet.apa.org/psycinfo/2004-17684-002
- jcoffland 10y ago
  Is it? I believe it's rather easy actually.

hiou 10y ago

Because optimizing for engagement is the modern equivalent of pandering?

derefr 10y ago
It'd be interesting to optimize for other factors. I'd like to see a news source that tries to optimize its presentation to incite the least possible controversy/argument while keeping engagement high. I'd guess this would be done by thoroughly explaining "both sides" of an issue (not in "equal times" terms, but in "all that can be said, has been said" terms)—but who knows, maybe there's a clever trick to doing something like this that such an algorithm would force into existence, much like PageRank forced the SEO industry into existence.
mjfl 10y ago
My comment assumed we were optimizing to minimize bias - which people do care about - and gave an approach to solve that. Optimizing for engagement is fine, and profitable. You just can't claim to be unbiased.

derefr 10y ago

I have a hypothesis that a lot of articles "bury the lede" of their political affiliation (both from ML algorithms and from people) because journalists believe that they can "save the world" by tricking people on one side of an issue into reading arguments on the other side of that issue. Whereas—they believe—if such articles are clearly marked, people on the "other side" will just avoid them and stay happily in their filter-bubbles.

majormajor 10y ago

> take abortion, you are either for choice, or you believe it is infanticide Well, no, even on abortion there is usually a position somewhere in the middle is where a lot of people end up (and you left out part of the spectrum, where even contraceptives are believed to be morally wrong), and usually the law has ended up somewhere in the middle, too. We confuse "loudest" views with "most common" a lot these days, so I do agree that promoting the most extreme views possible to drive "engagement" (a tactic figured out by people looking for votes long before the internet was even around to introduce engagement->clicks->dollars) is a bad behavior for aggregators, but that's a hard argument to make for a business as long as we hold the current view of short-term "fiduciary duty" ruling all.

humanrebar 10y ago
Some "middle positions" on abortion tend to be poorly thought out from a deontological perspective. Allowing abortion in cases of rape makes zero sense if abortion is murder, for example. Help should surely be given to rape victims, but giving it in the form of infanticide?
- dhimes 10y ago
  I understand your point but I don't think we should continue on the particular topic of abortion in this thread.
Lawtonfogle 10y ago
When considering all abortions there are many positions which limit different subsets as being wrong, but for any given situation one must choose between abortion allowed or abortion banned (and technically the choice of abortion required but that almost never pops up). So while one might be against partial birth abortion but for second trimester abortions, they still have to look at a given situation and say yay or nay.

mnarayan01 10y ago

take abortion, you are either for choice, or you believe it is infanticide The mere existence of the term "infanticide" suggests that the spectrum of opinions is less binary than you're making it out to be.

JumpCrisscross 10y ago

> take abortion, you are either for choice, or you believe it is infanticide I don't. It's fundamentally a question about when sex cells become a person, and if rights accumulate sequentially or instantaneously. There are lots of lines one can reasonably draw, e.g. independent viability (downside: disenfranchises premature babies), consciousness (no way to measure it), neural activity (how much?), heartbeat (why not if the tongue's wiggling?), conception, birth, et cetera. Breaking down problems is one way to help dis-engage tribal instincts.

mjfl 10y ago
It is my perception that people with the middle views you describe would still be considered "pro-choice" and would largely vote "pro-choice" but perhaps I am wrong. If those people were split 50/50 in the way that they vote I would concede that there is a middle ground in that issue.
- deleted 10y ago
  [deleted]

tuna-piano 10y ago

Should they also seek to hire neo-nazis to get the nazi viewpoint as well?

dlss 10y ago
Downvoted you because I don't quite see how this connects with GP other than a by being a Godwin laden form of "I'm not going to explain why, but trust me you're wrong" Worth a read IMO: http://www.paulgraham.com/disagree.html http://www.paulgraham.com/disagree.html
ommunist 10y ago
by the way, very interesting, where can I get that viewpoint?
venomsnake 10y ago
Yes of course. After all we strife for diversity, accrptance and tolerance. Unless we actually strife for cultural hegemony snd eradicating the wrong thoughts from the people that are currently losing the discourse.

leepowers 10y ago

> You're going to have to break the Turing Test before you get good automated content curation. There will always be error. Yes, but even a competent AI curation system is going to make mistakes. The only relevant question is "what is an acceptable rate of error?". (Hint: 0% is not an acceptable answer.) We may be able to get to an acceptable rate (or may already be there) with or without AI-complete. > People are worried about bias ... Information aggregators should have their members clearly spell out their positions on popular issues This isn't a problem that can be solved with meta-data. An aggregator could write a beautiful, open-minded mission statement and still write trash. Hell, Google's unofficial mantra is "don't do evil". Well, ok then, there's nothing to worry about, Google has clearly spelled out their position, so there's no need to analyze or criticize their policies. A policy statement or any other meta-data is not going to solve this. As you correctly noted an AI that passes the TT may be one way to work around this. But I'm skeptical. Any such AI would be at least as complicated as human intelligence, including all the foibles and errors that entails. An AI without our cognitive hang-ups would be smarter and more capable than us. Which would not increase my trust, and possibly decrease it, as the AI would no doubt have a bias to curate stories in it's own favor, or that put it (and AI in general) in the best light. Bias is an unavoidable component of any curation.

PhasmaFelis 10y ago
> The only relevant question is "what is an acceptable rate of error?". (Hint: 0% is not an acceptable answer.) Certainly it is. That just means you're admitting that the problem cannot be satisfactorily solved by current AI and must be left to humans for now.
- Sacho 10y ago
  I doubt the error rate was 0% with human curators. In fact, the article itself does not compare the error rate of the human curators to algorithm, just examines that it was above zero after the humans were removed. I think an acceptable statement to make would be that the human curators probably reduced the number of "fake stories", but that's not why they were removed. It's a different judgement call whether the trending feed is "better" or "worse" in some holistic sense. Furthermore, there seems to be an implicit assumption that Facebook's Trending stories should not contain "fake stories", but is that the intended goal of the service? I actually have no idea, since I've never used it.

unityByFreedom 10y ago

> there should be "block designed" hiring around these issues to maximize political diversity and reduce bias in aggregate How do we deal with those who would push treasonous information? If WikiLeaks publishes correspondence between, say, USA and China, that is private because China wanted it that way, and that hurts American allies, should this be allowed to trend on an American-based website? Has WikiLeaks contributed to the right wing propaganda we see being pushed online? Should some of this be punishable?

dlss 10y ago
These are very good questions, and have been debated at extensively (as you might imagine for such important questions) by humanity's legal, ethical, and philosophical scholars. Here's the current state of things with references as to why: https://en.wikipedia.org/wiki/Freedom_of_speech_in_the_United_States https://en.wikipedia.org/wiki/Freedom_of_speech_in_the_Unite... although I have some emotional disagreement with those policies at times, I can't say I'm well read enough on the topic to really have a strong disagreement personally.
- unityByFreedom 10y ago
  Thanks. I mention it because, as a tech enthusiast, I was once very much in the camp of "we should know everything our government does" Now, I understand that "we can't know everything, because some other governments won't tell us their true feelings if we don't keep those conversations private." Perhaps one day some Chinese or Russian hackers will level the playing field. In the mean time, I wonder if more tech folks can understand that our government might actually want to prosecute Snowden, Assange etc. for legitimate threats to our national security.

FullMtlAlcoholc 10y ago

>Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide. So let's stop pretending to be unbiased. You're falling into the trap of American politics by making every issue a binary choice. For instance, I find the abortion debate to be such a non-issue and if someone asks me what my stance is, I respond by saying I haven't done enough research or soul-searching to form a coherent, rational position on this issue. And I probably will not ever get around to doing this research. I'm ok not having a position on certain issues. I wish this was more acceptable. Instead, the US political duopoly and it's first to the post voting encouraged big tent parties who more or less have to agree on most issues. I'm going to paraphrase Rumsfeld's brilliant turn of phrase here, there are known unknowns, and there's power in accepting that.

lucio 10y ago
You should note that you have the luxury to even consider the issue of abortion because you've already been born.
- snovv_crash 10y ago
  Wow, that's a pretty dim view of humanity you have there. They were also conceived. Should contraceptives be illegal too? Just because there is a selection bias doesn't mean that the bias results in specifically positive or negative outcome. I could argue that because you were born you are against abortion because you were born too, and my argument would be just as valid. Of course I'm not making that argument, because it is just as silly as the one you are making.
- leohutson 10y ago
  They are also lucky their parents met, so your point is?
  - hueving 10y ago
    Tends to be a less inflammatory 'pro life' way of saying, "it's easy to be undecided when the decision isn't about you being murdered." Or something provocative along those lines.

artimaeis 10y ago

> Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide. Hogwash. First, there's the spectrum of finding where human life begins. There's many stances here from at the point of conception to post-birth, and everywhere between. Then there's the factor of a woman's right to her body and life. Given the first factor - we have a whole new complex spectrum here where we weigh in the life of the infant vs the life of the mother. To further complicate this, there's the woman's right to privacy. This was the basis by which Roe v Wade was decided in the first place. So there's at least 3 spectrum on which any citizen can find themselves in the middle of in that issue. I agree that humans probably don't have the ability to be politically unbiased. I'm just not sure that bias matters that much. But to pretend that there are issues without middle ground is to fall into the trap that so many groups face these days - that divisive and non-empathetic stance will tear at us and divide us further. There's simply no place for it.

Bartweiss 10y ago

The Turing Test thing was my first reaction also. These are sites consciously imitating the stylistic markers of real news, where the only signal for falsification is the content of the story. They regularly fool real people - sometimes because The Onion's sarcasm slips past someone, sometimes because the piece is 100% plausible, yet false. Facebook could save itself some predictable pain by blacklisting The Onion and SportsPickle and other high-profile fake news sites, but they aren't going to be able to crack this problem with algorithms (or if they do, those algorithms will have far better uses).

mrob 10y ago

>Many modern political issues do not have a middle ground - take abortion, you are either for choice, or you believe it is infanticide Those two options are not mutually exclusive. All healthy humans have an instinctive aversion to murder, and murder is universally considered morally wrong, but they do not have such an aversion to infanticide. In many cultures infanticide has been considered acceptable, and even in modern American people aren't going to ask too many questions if a new mother claims it was an accident. A new-born baby does not speak, it does not use tools, it fails the mirror test. By any objective measure it's less of a person than many animals we kill without a second thought. There's a strong argument that the mother should be allowed to make that decision, possibly even up to a short time (eg. 24 hours) post-birth.