10 ms·

If a poem (or book) makes 10% of its readers more likely to become geniuses and contribute to solving world problems such as cancer, but 0.1% of its readers are

by vladd 5y ago

If a poem (or book) makes 10% of its readers more likely to become geniuses and contribute to solving world problems such as cancer, but 0.1% of its readers are more likely to commit suicide, should that book be banned by law? Today's online society is based on posts created by content creators around the world, where algorithms can barely scratch the surface at interpreting their content, humans don't scale in reviewing every post, but statistics such as the above could be arguably inferred easily based on a combination of engagement (click/scrolls) data and attrition/session-revisits numbers. Which is really problematic, because codifying into law rules and punishments based on aggregated outcomes and impact to us as a society (or to society sub-segments such as teens) makes it a very hard process to navigate between censorship vs. positive overall outcome vs. specific negative outcome on some outliers.

cutemonster 5y ago
It's a misleading comparison. From what I read, trafficking / sex slavery was (is) happening via some places on FB, the company knew about it, did nothing. For example. Edit: the article from this HN discussion: https://news.ycombinator.com/item?id=28741532 https://news.ycombinator.com/item?id=28741532 , search for "traffickers" and "drug cartels". /Edit. Understaffed moderation teams, although FB had lots of money
- vladd 5y ago
  Let's try to phrase it in an actionable way for the law-makers to act upon it. Are you suggesting that any profitable company hosting user-submitted content should invest all the profits in moderation teams to the point where they are either a) becoming profit-neutral or b) all the relevant content has been reviewed by a human moderator? And how do you define relevant content -- having had 50 views? 10 views? 1 view? Who should decide where to set these limits? Do we believe politicians are going to do a better job at it rather than the existing situation? Or should we ban any non-human reviewed post just to move the certainty of illegal posts removals from 99.9% to 99.99%? (humans do make mistakes too) (Facebook is really big so having just 99.99% of posts in compliance still means an awful amount of them escaping the system undetected)
  - tsimionescu 5y ago
    > Are you suggesting that any profitable company hosting user-submitted content should invest all the profits in moderation teams to the point where they are either a) becoming profit-neutral or b) all the relevant content has been reviewed by a human moderator? Yes, obviously. Why should a company get to profit from sex traffic or any other such content on their platform, just because it would cost money to take it down?
- dfadsadsf 5y ago
  I know that somebody is raping someone in NYC right now and somebody will be killed in Chicago by the end of the day today. Should we ban the cities or at least force them to spend all their budget on security? Or set up curfew for citizens? May be public hanging a la Taliban - those definitely reduce crime. Humans are using FB and where you have humans they commit crimes. Trying to eradicate all crime when you have humans in the loop is generally not great idea. Besides fighting trafficking/sex slavery with very few exceptions generally means harassing women with zero benefit to society or reduction in actual sex crimes.
  - freetinker 5y ago
    But because we know humans rape and kill, we take measures to create circumstances that reduce the probability of such things happening. Such as well-lit streets or gun control laws.
- haroldp 5y ago
  Sex slavery is being facilitated by telephone conversations. What is the phone company's obligation to do about that?
  - finfinfin 5y ago
    Would you agree that it would be wrong for telephone companies to amplify sex slavery conversations? Like they would call you directly and just let you participate in the conversation because that would generate more engagement?
    - haroldp 5y ago
      That is a very good counter point. I haven't read this facebook story yet, but I am willing to assume for argument that describes what happened. I guess it would depend for me on whether people saw sex-slavery content and decided to amplify it, vs an algorithm that finds and promotes "engaging" things without being very smart about what they are.
    - vladd 5y ago
      When phone companies came into existence, that's exactly what they did -- they amplified such conversations by making it easier for people to have phone calls and talk at a distance of each other. They also got amplified whenever long distance calls got cheaper (as the overall volume of conversations increased).
    - Dracophoenix 5y ago
      How are you defining "amplification"? Phones already operate by complex signal amplification over long distances. Why do you think burner phones are still prevalent for all manner of illicit activity? I don't think the phone company should be shut down because others can use it in a way that's considered devious. I don't think the phone company should play "morality police" either. I simply expect the phone company to simply provide the service I paid for. This type of thinking strikes as the kind that would damn Gutenberg for inventing the movable-type printing press because print has been used to disseminate propaganda and debauchery to billions of people several centuries later.
drdeca 5y ago
if magic books were real, then the way we would have to treat books would be much different.
finfinfin 5y ago
Looks like you are willfully ignoring Facebook’s own findings. They know that polarizing content is more engaging yet harmful… and they choose to amplify it anyway. The same old argument that it’s hard therefore let’s not do anything is not applicable. Facebook is not a neutral platform that just shows all posts from your friends in a chronological order. They are actively manipulating the stream and are fully responsible for what you consume.
- vladd 5y ago
  Walmart is "manipulating" the placement of products on the shelf so that it's more likely for you to engage in bulk buying when you visit their stores. Both Facebook and Walmart have a fiduciary duty to their shareholders to create value for them. The difference is that, with user generated content, the idea of black and white "bounds" of the law is no longer applicable and you have to devise a system of checks and balances based on probabilities. You can consider 10'000 posts for offline analysis: give them to some human raters and decide retrospectively what engagement and thoughts (positive/negative) are they generating in teens, which should enable you to draw some statistics about the expected average outcome. This doesn't mean it's either scalable or economically feasible to do so in real time for every post (so you cannot take decisions based on something that doesn't exist at the individual post level). You can have multiple algorithms, send all of them to human raters and get for each algorithm some aggregated behaviour, but then we're back to the book question above -- what ratio of positive vs negative outcome in outliers is acceptable, and how do you define a "legal"/"allowed" algorithm?
  - chalst 5y ago
    My regular reminder that there is no fiduciary duty to behave unethically. Fiduciary duty is a class of highly specific legal obligations on directors to act attentively and not put their own financial interests above those of shareholders. It is not an obligation to maximise return on investment. Cf. https://news.ycombinator.com/item?id=20776770 https://news.ycombinator.com/item?id=20776770
  - finfinfin 5y ago
    I am baffled by this display of lack of ethics. Do we need a Walmart comparison to put Facebook’s action in perspective? Facebook - by its own acknowledgement - negatively affects teenage mental health and the democratic processes in many countries. Do you see how different this is from selling more mayonnaise jars in Walmart? Facebook doesn’t have a duty to manipulate content. This is a very weak excuse that works mostly for people directly benefiting from the situation. Didn’t cigarette companies have a duty to maximize profits? Pharma companies pushing accessible opioids? Is that a more apt analogy?
haroldp 5y ago
Please stop down-voting thoughtful comments such as this just because you disagree with them.
jimkleiber 5y ago
> If a poem (or book) makes 10% of its readers more likely to become geniuses and contribute to solving world problems such as cancer, but 0.1% of its readers are more likely to commit suicide, should that book be banned by law? I really don't know the answer. I've struggled with this tradeoff myself, as I've built some tools that have powerfully impacted people on an emotional level and I've been hesitant to put them out there because of the severe damage they might do to a small percentage of the population. That being said, I read a few essays a few years back about Frankenstein and this from a Q&A[0] in the same Slate series[1] that I try to remember when I think about creating such tools: > Does that make it into a warning against playing God? > It’s probably a mistake to suggest that the novel is just a critique of those who would usurp the divine mantle. Instead, you can read it as a warning about the ways that technologists fall short of their ambitions, even in their greatest moments of triumph. > Look at what happens in the novel: After bringing his creature to life, Frankenstein effectively abandons it. Later, when it entreats him to grant it the rights it thinks it deserves, he refuses. Only then—after he reneges on his responsibilities—does his creation really go bad. We all know that Frankenstein is the doctor and his creation is the monster, but to some extent it’s the doctor himself who’s made monstrous by his inability to take responsibility for what he’s wrought. I try to remind myself of this lesson that perhaps it's not about not creating powerful things but trying to continue to maintain these things instead of abandoning them and just accepting the havoc they wreaked. I think where I often feel the most frustrated is in believing that FB doesn't really seem to me to be trying that hard in 1) making the platform less addictive, 2) getting rid of bots, 3) suggesting which legislation they want (instead of just punting and say "we said we want regulation, it's your job Congress to create it"), etc. I don't think they'll ever get rid of all the things that cause harm, as it's even hard to choose dinner for a party of 4 where someone won't get hurt or angry, and this is a scale almost 1 billion times larger. I just want to have the impression that they are trying, or at the bare minimum, that they have the courage to say that sometimes bad things come with the good and they openly say that they are choosing that tradeoff. Maybe they've said it that way, I just don't seem to trust them much in terms of trying to take responsibility for their creation. [0]: https://slate.com/technology/2017/01/why-frankenstein-is-still-relevant-almost-200-years-after-it-was-published.html https://slate.com/technology/2017/01/why-frankenstein-is-sti... [1]: https://slate.com/tag/futurography-frankenstein https://slate.com/tag/futurography-frankenstein *edit: formatting line breaks

by vladd 5y ago

cutemonster 5y ago

It's a misleading comparison. From what I read, trafficking / sex slavery was (is) happening via some places on FB, the company knew about it, did nothing. For example. Edit: the article from this HN discussion: https://news.ycombinator.com/item?id=28741532 https://news.ycombinator.com/item?id=28741532 , search for "traffickers" and "drug cartels". /Edit. Understaffed moderation teams, although FB had lots of money

vladd 5y ago
Let's try to phrase it in an actionable way for the law-makers to act upon it. Are you suggesting that any profitable company hosting user-submitted content should invest all the profits in moderation teams to the point where they are either a) becoming profit-neutral or b) all the relevant content has been reviewed by a human moderator? And how do you define relevant content -- having had 50 views? 10 views? 1 view? Who should decide where to set these limits? Do we believe politicians are going to do a better job at it rather than the existing situation? Or should we ban any non-human reviewed post just to move the certainty of illegal posts removals from 99.9% to 99.99%? (humans do make mistakes too) (Facebook is really big so having just 99.99% of posts in compliance still means an awful amount of them escaping the system undetected)
- tsimionescu 5y ago
  > Are you suggesting that any profitable company hosting user-submitted content should invest all the profits in moderation teams to the point where they are either a) becoming profit-neutral or b) all the relevant content has been reviewed by a human moderator? Yes, obviously. Why should a company get to profit from sex traffic or any other such content on their platform, just because it would cost money to take it down?
dfadsadsf 5y ago
I know that somebody is raping someone in NYC right now and somebody will be killed in Chicago by the end of the day today. Should we ban the cities or at least force them to spend all their budget on security? Or set up curfew for citizens? May be public hanging a la Taliban - those definitely reduce crime. Humans are using FB and where you have humans they commit crimes. Trying to eradicate all crime when you have humans in the loop is generally not great idea. Besides fighting trafficking/sex slavery with very few exceptions generally means harassing women with zero benefit to society or reduction in actual sex crimes.
- freetinker 5y ago
  But because we know humans rape and kill, we take measures to create circumstances that reduce the probability of such things happening. Such as well-lit streets or gun control laws.
haroldp 5y ago
Sex slavery is being facilitated by telephone conversations. What is the phone company's obligation to do about that?
- finfinfin 5y ago
  Would you agree that it would be wrong for telephone companies to amplify sex slavery conversations? Like they would call you directly and just let you participate in the conversation because that would generate more engagement?
  - haroldp 5y ago
    That is a very good counter point. I haven't read this facebook story yet, but I am willing to assume for argument that describes what happened. I guess it would depend for me on whether people saw sex-slavery content and decided to amplify it, vs an algorithm that finds and promotes "engaging" things without being very smart about what they are.
  - vladd 5y ago
    When phone companies came into existence, that's exactly what they did -- they amplified such conversations by making it easier for people to have phone calls and talk at a distance of each other. They also got amplified whenever long distance calls got cheaper (as the overall volume of conversations increased).
  - Dracophoenix 5y ago
    How are you defining "amplification"? Phones already operate by complex signal amplification over long distances. Why do you think burner phones are still prevalent for all manner of illicit activity? I don't think the phone company should be shut down because others can use it in a way that's considered devious. I don't think the phone company should play "morality police" either. I simply expect the phone company to simply provide the service I paid for. This type of thinking strikes as the kind that would damn Gutenberg for inventing the movable-type printing press because print has been used to disseminate propaganda and debauchery to billions of people several centuries later.

drdeca 5y ago

if magic books were real, then the way we would have to treat books would be much different.

finfinfin 5y ago

Looks like you are willfully ignoring Facebook’s own findings. They know that polarizing content is more engaging yet harmful… and they choose to amplify it anyway. The same old argument that it’s hard therefore let’s not do anything is not applicable. Facebook is not a neutral platform that just shows all posts from your friends in a chronological order. They are actively manipulating the stream and are fully responsible for what you consume.

vladd 5y ago
Walmart is "manipulating" the placement of products on the shelf so that it's more likely for you to engage in bulk buying when you visit their stores. Both Facebook and Walmart have a fiduciary duty to their shareholders to create value for them. The difference is that, with user generated content, the idea of black and white "bounds" of the law is no longer applicable and you have to devise a system of checks and balances based on probabilities. You can consider 10'000 posts for offline analysis: give them to some human raters and decide retrospectively what engagement and thoughts (positive/negative) are they generating in teens, which should enable you to draw some statistics about the expected average outcome. This doesn't mean it's either scalable or economically feasible to do so in real time for every post (so you cannot take decisions based on something that doesn't exist at the individual post level). You can have multiple algorithms, send all of them to human raters and get for each algorithm some aggregated behaviour, but then we're back to the book question above -- what ratio of positive vs negative outcome in outliers is acceptable, and how do you define a "legal"/"allowed" algorithm?
- chalst 5y ago
  My regular reminder that there is no fiduciary duty to behave unethically. Fiduciary duty is a class of highly specific legal obligations on directors to act attentively and not put their own financial interests above those of shareholders. It is not an obligation to maximise return on investment. Cf. https://news.ycombinator.com/item?id=20776770 https://news.ycombinator.com/item?id=20776770
- finfinfin 5y ago
  I am baffled by this display of lack of ethics. Do we need a Walmart comparison to put Facebook’s action in perspective? Facebook - by its own acknowledgement - negatively affects teenage mental health and the democratic processes in many countries. Do you see how different this is from selling more mayonnaise jars in Walmart? Facebook doesn’t have a duty to manipulate content. This is a very weak excuse that works mostly for people directly benefiting from the situation. Didn’t cigarette companies have a duty to maximize profits? Pharma companies pushing accessible opioids? Is that a more apt analogy?

haroldp 5y ago

Please stop down-voting thoughtful comments such as this just because you disagree with them.

jimkleiber 5y ago

> If a poem (or book) makes 10% of its readers more likely to become geniuses and contribute to solving world problems such as cancer, but 0.1% of its readers are more likely to commit suicide, should that book be banned by law? I really don't know the answer. I've struggled with this tradeoff myself, as I've built some tools that have powerfully impacted people on an emotional level and I've been hesitant to put them out there because of the severe damage they might do to a small percentage of the population. That being said, I read a few essays a few years back about Frankenstein and this from a Q&A[0] in the same Slate series[1] that I try to remember when I think about creating such tools: > Does that make it into a warning against playing God? > It’s probably a mistake to suggest that the novel is just a critique of those who would usurp the divine mantle. Instead, you can read it as a warning about the ways that technologists fall short of their ambitions, even in their greatest moments of triumph. > Look at what happens in the novel: After bringing his creature to life, Frankenstein effectively abandons it. Later, when it entreats him to grant it the rights it thinks it deserves, he refuses. Only then—after he reneges on his responsibilities—does his creation really go bad. We all know that Frankenstein is the doctor and his creation is the monster, but to some extent it’s the doctor himself who’s made monstrous by his inability to take responsibility for what he’s wrought. I try to remind myself of this lesson that perhaps it's not about not creating powerful things but trying to continue to maintain these things instead of abandoning them and just accepting the havoc they wreaked. I think where I often feel the most frustrated is in believing that FB doesn't really seem to me to be trying that hard in 1) making the platform less addictive, 2) getting rid of bots, 3) suggesting which legislation they want (instead of just punting and say "we said we want regulation, it's your job Congress to create it"), etc. I don't think they'll ever get rid of all the things that cause harm, as it's even hard to choose dinner for a party of 4 where someone won't get hurt or angry, and this is a scale almost 1 billion times larger. I just want to have the impression that they are trying, or at the bare minimum, that they have the courage to say that sometimes bad things come with the good and they openly say that they are choosing that tradeoff. Maybe they've said it that way, I just don't seem to trust them much in terms of trying to take responsibility for their creation. [0]: https://slate.com/technology/2017/01/why-frankenstein-is-still-relevant-almost-200-years-after-it-was-published.html https://slate.com/technology/2017/01/why-frankenstein-is-sti... [1]: https://slate.com/tag/futurography-frankenstein https://slate.com/tag/futurography-frankenstein *edit: formatting line breaks