6 ms·

They've also said there will be more harmful (but legal) content on there as they'll no longer automatically look for it, but require it to be reported before t

They've also said there will be more harmful (but legal) content on there as they'll no longer automatically look for it, but require it to be reported before taking action. As someone who worked on harmful content, specifically suicide and self injury, this is just nuts - they were raked over the coals in both the UK by an inquest into the suicide of a teenage user who rabbit holed on this harmful content, and also with the parents of teenagers who took their lives, who Zuck turned around and apologised to as his latest senate hearing. There is research that shows exposure to suicide and self injury content increases suicidal ideation. I'm hoping that there is some nuance that has been missed from the article, but if not, this would seem like a slam dunk for both the UK and EU regulators to take them to task on.

rayiner 2y ago
[flagged]
- ceejayoz 2y ago
  > And why aren't you considering the countervailing harm to society of centralized moderation? Are there reproducible studies showing that? What is the effect size? (These objections go both ways!)
  - rayiner 2y ago
    There’s lots of mainstream media content I think is psychologically harmful and should be suppressed, such as content normalizing adultery. But I’m quite content to live in a society where the social norm favors people saying what they want and the burden is on the opponents of that to produce strong evidence of harm.
    - ceejayoz 2y ago
      Convenient that only your debate opponents need provide reproducible studies that meet your standards.
      - layer8 2y ago
        No, this is the “presumed innocent until proven guilty” principle.
        ceejayoz 2y ago
        Why does that principle not apply to “moderation is bad”?
        threatofrain 2y ago
        But all we see are two proponents in a civil trial. Shouldn't the standard be the well known "preponderance of evidence"? Though personally preponderance of evidence seems to be a shitty standard too because I might be listening to two awful theories and be forced to conclude one is the winner. Theories should rise above a minimum threshold to even consider sniffing at before we consider one as superior over the other.
        layer8 2y ago
        I agree that there needs to be a better standard than just “more likely than not”. Freedom of expression is a fundamental good, and there should be clear evidence of harm outweighing that good, before curtailing it. Regarding my previous comment, my intent was to point out the GP comment’s position (because the parent’s comment seemed to be beside the point), not necessarily to endorse it.
    - myvoiceismypass 2y ago
      Which studies have you read that show the psychological harm of seeing adulterous content?
  - n4r9 2y ago
    Right. The challenge for free speech absolutists is to demonstrate that free speech takes priority over moderating hate speech, adult content, highly addictive media etc... . That demonstration needs to evidence-based and framed in terms of short and long term social harms/impact. Simply saying "censorship hasn't gone well for some countries" or "having a free speech zone is extremely important to the future of civilization" is not enough.
    - nradov 2y ago
      No, that's not how this works. By default everything permitted. The entire burden of evidence rests on those who want bans or restrictions.
      - n4r9 2y ago
        Hmm. Who decides "how this works"? Consider hate speech. There is a clear short-term benefit of moderation: reducing the harms to marginalised people from being exposed to threats to their person, identity, and way of life. In the face of this benefit, the absolute free speech advocate must provide a counter-argument for why free speech overrides that harm-reduction.
        logicchains 2y ago
        >reducing the harms to marginalised people from being exposed to threats to their person, identity, and way of life This only makes sense if you use a recent definition of "harm" created by censorship advocates that's divorced from the traditional meaning. In criminal law, harm traditionally (and still does in America) mean actually physically harming someone's body or making threats to do so. Censorship advocates are the ones making the claim that mere words should also constitute harm, so the onus is on them to justify why they want to change the meaning of the word like that.
        ceejayoz 2y ago
        > In criminal law, harm traditionally (and still does in America) mean actually physically harming someone's body or making threats to do so. Fraud can be criminal, without bodily harm or threats. Verbal child abuse can be criminal, without bodily harm or threats. There are lots of criminal harms not covered by your claimed definition in the American legal system.
  - mike_hearn 2y ago
    Arguably they don't go both ways. The distinction is action vs inaction. If someone wants action from someone else, they need to argue the case for that action. Arguing for not doing something on the other hand is never necessary. To see why, consider that the space of possible actions someone could not take is infinite. If there's an expectation that someone do a ton of research and work to argue for why they are not doing something, then the amount of work they would have to do is thus also infinite. This way lies madness, which is why in reality the default outcome that results from not acting is always taken as a given. Sometimes this reality is obfuscated by activists. They find some group of people who are just doing their thing, and demand that those people do some extra things (usually some costly things). The arguments they make for this are weak, but when the targeted people say they'd rather not do those extra things the activists demand their targets argue for not doing what the activists want to whatever level of effort (or greater) they themselves made. This can be an effective bullying tactic but isn't legitimate: it's on those who want action to argue the case for it, not those who don't to argue against. Digital platforms like social networks default to uncensored. If the operators do nothing, then by the way they are built content is allowed. It takes additional work to categorize posts and block certain kinds of content. So the default outcome is free speech. If someone wants someone else to do work to suppress that, then it's on them to prove that it's truly necessary and that the benefits outweigh the harms. But that doesn't cut both ways; it's not required for other people to take on the argument for free speech. That's the default outcome so it just wins by default if the other side can't prove their case to a sufficiently convincing level.
    - ceejayoz 2y ago
      Choosing not to act is an action. The choice not to moderate certain content is a choice to permit certain content.
      - mike_hearn 2y ago
        Not acting is, by definition, not an action.
        2y ago
- BryantD 2y ago
  Enjoy: https://www.nsrf.ie/wp-content/uploads/2023/09/Harmful-impact-of-suicide-and-self-harm-content-online-Review-of-the-literature-Update-August-2023-Final.pdf https://www.nsrf.ie/wp-content/uploads/2023/09/Harmful-impac...
- philjohn 2y ago
  Are we arguing that graphic images of suicide and self injury are required for open discourse?
  - zahlman 2y ago
    Freedom of speech is not about what's "required". That's why pornography is allowed to exist.
    - philjohn 2y ago
      And Facebook don't allow pornography either. What point are you trying to make here?
- miltonlost 2y ago
  Showing pictures of suicide is "open discourse" now? That's what you're defending?
BryantD 2y ago
This exactly mirrors my thoughts, although I don't work in your field. One quote: "For example, in December 2024, we removed millions of pieces of content every day. While these actions account for less than 1% of content produced every day, we think one to two out of every 10 of these actions may have been mistakes (i.e., the content may not have actually violated our policies)." That is first order data and it's interesting. However, before making policy decisions, I would want the second order data: what is the human cost of those mistakes, and what percentage of policy-violating content will not be removed as a result of these changes? Finally, what's the cost of not removing that percentage? For that matter, by talking about the percentage of active mistakes without saying how many policy violations are currently missed, you're framing the debate in a certain direction.
- philjohn 2y ago
  Indeed. The human cost of a piece of content being taken down depends on the piece of content, and the reason behind posting it. In the case of someone posting about recovery from self injury and including a photo of their healed self-harm scars, having that taken down by mistake would be more harmful than someone who posted a cartoon depiction of suicide for the lolz.
  - BryantD 2y ago
    Yes. My personal belief, for whatever that's worth, is that communication and speech are one of the most powerful tools any of us have. Talking can change minds, move societies, arouse emotions, and in general makes a difference. This is true no matter the format (text, voice, etc.). That means that restricting communication should not be a casual activity. Free speech is a good ideal for a reason. It also means that, if you believe in the primacy of free speech, you are obligated to consider the implications of that belief. Speech has effects. In my adult life, since 1990, we have seen a major change in the ease of communication. IMHO, society hasn't been able to fully adjust to that change -- or rather, that huge suite of changes. I sincerely do not know what a healthy society using the Internet looks like; I don't think we're in one now. All of these arguments (on all sides, mine included) are hampered by our lack of perspective.
naasking 2y ago
> There is research that shows exposure to suicide and self injury content increases suicidal ideation. Yes. However, I find this obsession with harm-based value judgment to the exclusion of all other considerations ethically problematic, to put it mildly. Ethics does not reduce solely to considerations of harm.
- philjohn 2y ago
  Would you mind expanding on that please, what are the ethically problematic things you are trying to balance against this?
  - naasking 2y ago
    I reject the implicit assumption that Zuckerberg has an ethical duty to reduce indirect harm from hosting content, and I also reject the implicit assumption that ethical calculations ("balancing" harm) are a meaningful way to reason about ethics. I think both of these assumptions are ethically problematic. The only reason Zuckerberg apologized to parents is because other people would have punished him had he not, which does not imply sound ethics or that he agrees with what he said. I also think normalizing an infrastructure whose sole purpose is to suppress speech is ethically problematic. Rights are rarely expanded once taken. You're right that people post abhorrent stuff on such sites, but I prefer approaches like filters over suppression. I also think that if reducing indirect harm is important to certain people (like reducing suicide), then there are known effective ways that don't require coercive power, like the public campaigns against smoking and MADD which have both had significant impacts.
    - philjohn 2y ago
      In what way is not recommending content, or demoting it in feeds, coercive? I'm sorry, but maximalist free speech positions fail in the face of reality.
  - crindy 2y ago
    Freedom of expression comes to mind. If someone had a friend commit suicide, should they not be able to discuss their experience in public?
Dig1t 2y ago
“Think of the children” isn’t really a good argument for censoring completely legal political discourse, which is what has been happening. They are admitting that there has been a global push against free speech on these platforms. >There is research that shows exposure to suicide and self injury content increases suicidal ideation. I mean do you really need research to show this link? Of course it does. We are okay with slapping an “R” rating on movies and allowing parents to be the ones who decide what content their kids can see. Why can’t we decide that parents also need to be the ones to stop their kids from consuming bad content on social media?
- philjohn 2y ago
  Automatically demoting, not recommending and adding "mark as disturbing" screens is what's going away - which is akin to the "R" rating. But at this point, I'm siding with the "no social media for adolescents" people more and more.
jarsin 2y ago
I've already seen disturbing stuff on X since Elon took over that I never would have seen when it was twitter. They don't even show the warning "this might be harmful content" on images and videos anymore. The X algo seems to go haywire every couple of days and dumps a bunch of this crap in my feed until I block 20+ bluecheck accounts showing this crap. I believe it's only going to get worse going forward as they all adopt these policies.
- jeromegv 2y ago
  It’s okay to leave X.
- biosboiii 2y ago
  My profile is largely unused, I follow no one, and like 1/3 times I open up the front page I get straight holocaust denial threads suggested. Completely insane.
dado3212 2y ago
> So, we’re going to continue to focus these systems on tackling illegal and high-severity violations, like terrorism, child sexual exploitation, drugs, fraud and scams. I don't think this is exhaustive, and I think SSI (suicide/self-injury) + ED/etc. stuff is considered high-severity.
- philjohn 2y ago
  Fingers crossed.
zahlman 2y ago
Counterpoint: censorship inherently harms everyone. People I follow on Youtube have repeatedly had their ability to discuss topics such as suicide seriously interfered with. It actually gets in the way of factual reporting when a suicide occurs in the community and of discussing the facts of the situation so that people can learn from it and possibly prevent future deaths. Not to mention, people just straight up have a right to talk about these things. It is not moral to hold one person responsible for an unintended and not reasonably foreseeable reaction to the discussion. And joking about these topics is legitimately therapeutic for some.
- philjohn 2y ago
  I'm not talking about that here - and that always fell under protecting voice - if mistakes were made they should have been reversed on appeal. e.g. imagery of healed scars in the context of recovery, discussions of struggles with mental health, suicidal ideation etc. I'm talking about graphic images of self harm, suicide, eating disorders. And at some point you have to weigh the maximalist interpretation of free speech "you have to host whatever I want, as long as it's not illegal" with "promoting this stuff causes active harm, no".
  - zahlman 2y ago
    >And at some point you have to weigh the maximalist interpretation of free speech "you have to host whatever I want, as long as it's not illegal" with "promoting this stuff causes active harm, no". The burden of proof is on you to demonstrate that it causes such harm. I don't generally think people should be held responsible for the unintended reaction to their speech of a small minority of the audience.
    - philjohn 2y ago
      Having a piece of content removed, or demoted and not recommended, is being held responsible? Also as per the inquest into the death of Molly Russel found based on the preponderance of evidence, exposure to this kind of graphic content was largely the causative agent in her suicide. What would the bar you require be, is there a bar?

rayiner 2y ago

[flagged]

ceejayoz 2y ago
> And why aren't you considering the countervailing harm to society of centralized moderation? Are there reproducible studies showing that? What is the effect size? (These objections go both ways!)
- rayiner 2y ago
  There’s lots of mainstream media content I think is psychologically harmful and should be suppressed, such as content normalizing adultery. But I’m quite content to live in a society where the social norm favors people saying what they want and the burden is on the opponents of that to produce strong evidence of harm.
  - ceejayoz 2y ago
    Convenient that only your debate opponents need provide reproducible studies that meet your standards.
    - layer8 2y ago
      No, this is the “presumed innocent until proven guilty” principle.
      - ceejayoz 2y ago
        Why does that principle not apply to “moderation is bad”?
      - threatofrain 2y ago
        But all we see are two proponents in a civil trial. Shouldn't the standard be the well known "preponderance of evidence"? Though personally preponderance of evidence seems to be a shitty standard too because I might be listening to two awful theories and be forced to conclude one is the winner. Theories should rise above a minimum threshold to even consider sniffing at before we consider one as superior over the other.
        layer8 2y ago
        I agree that there needs to be a better standard than just “more likely than not”. Freedom of expression is a fundamental good, and there should be clear evidence of harm outweighing that good, before curtailing it. Regarding my previous comment, my intent was to point out the GP comment’s position (because the parent’s comment seemed to be beside the point), not necessarily to endorse it.
  - myvoiceismypass 2y ago
    Which studies have you read that show the psychological harm of seeing adulterous content?
- n4r9 2y ago
  Right. The challenge for free speech absolutists is to demonstrate that free speech takes priority over moderating hate speech, adult content, highly addictive media etc... . That demonstration needs to evidence-based and framed in terms of short and long term social harms/impact. Simply saying "censorship hasn't gone well for some countries" or "having a free speech zone is extremely important to the future of civilization" is not enough.
  - nradov 2y ago
    No, that's not how this works. By default everything permitted. The entire burden of evidence rests on those who want bans or restrictions.
    - n4r9 2y ago
      Hmm. Who decides "how this works"? Consider hate speech. There is a clear short-term benefit of moderation: reducing the harms to marginalised people from being exposed to threats to their person, identity, and way of life. In the face of this benefit, the absolute free speech advocate must provide a counter-argument for why free speech overrides that harm-reduction.
      - logicchains 2y ago
        >reducing the harms to marginalised people from being exposed to threats to their person, identity, and way of life This only makes sense if you use a recent definition of "harm" created by censorship advocates that's divorced from the traditional meaning. In criminal law, harm traditionally (and still does in America) mean actually physically harming someone's body or making threats to do so. Censorship advocates are the ones making the claim that mere words should also constitute harm, so the onus is on them to justify why they want to change the meaning of the word like that.
        ceejayoz 2y ago
        > In criminal law, harm traditionally (and still does in America) mean actually physically harming someone's body or making threats to do so. Fraud can be criminal, without bodily harm or threats. Verbal child abuse can be criminal, without bodily harm or threats. There are lots of criminal harms not covered by your claimed definition in the American legal system.
- mike_hearn 2y ago
  Arguably they don't go both ways. The distinction is action vs inaction. If someone wants action from someone else, they need to argue the case for that action. Arguing for not doing something on the other hand is never necessary. To see why, consider that the space of possible actions someone could not take is infinite. If there's an expectation that someone do a ton of research and work to argue for why they are not doing something, then the amount of work they would have to do is thus also infinite. This way lies madness, which is why in reality the default outcome that results from not acting is always taken as a given. Sometimes this reality is obfuscated by activists. They find some group of people who are just doing their thing, and demand that those people do some extra things (usually some costly things). The arguments they make for this are weak, but when the targeted people say they'd rather not do those extra things the activists demand their targets argue for not doing what the activists want to whatever level of effort (or greater) they themselves made. This can be an effective bullying tactic but isn't legitimate: it's on those who want action to argue the case for it, not those who don't to argue against. Digital platforms like social networks default to uncensored. If the operators do nothing, then by the way they are built content is allowed. It takes additional work to categorize posts and block certain kinds of content. So the default outcome is free speech. If someone wants someone else to do work to suppress that, then it's on them to prove that it's truly necessary and that the benefits outweigh the harms. But that doesn't cut both ways; it's not required for other people to take on the argument for free speech. That's the default outcome so it just wins by default if the other side can't prove their case to a sufficiently convincing level.
  - ceejayoz 2y ago
    Choosing not to act is an action. The choice not to moderate certain content is a choice to permit certain content.
    - mike_hearn 2y ago
      Not acting is, by definition, not an action.
      - 2y ago
BryantD 2y ago
Enjoy: https://www.nsrf.ie/wp-content/uploads/2023/09/Harmful-impact-of-suicide-and-self-harm-content-online-Review-of-the-literature-Update-August-2023-Final.pdf https://www.nsrf.ie/wp-content/uploads/2023/09/Harmful-impac...
philjohn 2y ago
Are we arguing that graphic images of suicide and self injury are required for open discourse?
- zahlman 2y ago
  Freedom of speech is not about what's "required". That's why pornography is allowed to exist.
  - philjohn 2y ago
    And Facebook don't allow pornography either. What point are you trying to make here?
miltonlost 2y ago
Showing pictures of suicide is "open discourse" now? That's what you're defending?

BryantD 2y ago

This exactly mirrors my thoughts, although I don't work in your field. One quote: "For example, in December 2024, we removed millions of pieces of content every day. While these actions account for less than 1% of content produced every day, we think one to two out of every 10 of these actions may have been mistakes (i.e., the content may not have actually violated our policies)." That is first order data and it's interesting. However, before making policy decisions, I would want the second order data: what is the human cost of those mistakes, and what percentage of policy-violating content will not be removed as a result of these changes? Finally, what's the cost of not removing that percentage? For that matter, by talking about the percentage of active mistakes without saying how many policy violations are currently missed, you're framing the debate in a certain direction.

philjohn 2y ago
Indeed. The human cost of a piece of content being taken down depends on the piece of content, and the reason behind posting it. In the case of someone posting about recovery from self injury and including a photo of their healed self-harm scars, having that taken down by mistake would be more harmful than someone who posted a cartoon depiction of suicide for the lolz.
- BryantD 2y ago
  Yes. My personal belief, for whatever that's worth, is that communication and speech are one of the most powerful tools any of us have. Talking can change minds, move societies, arouse emotions, and in general makes a difference. This is true no matter the format (text, voice, etc.). That means that restricting communication should not be a casual activity. Free speech is a good ideal for a reason. It also means that, if you believe in the primacy of free speech, you are obligated to consider the implications of that belief. Speech has effects. In my adult life, since 1990, we have seen a major change in the ease of communication. IMHO, society hasn't been able to fully adjust to that change -- or rather, that huge suite of changes. I sincerely do not know what a healthy society using the Internet looks like; I don't think we're in one now. All of these arguments (on all sides, mine included) are hampered by our lack of perspective.

naasking 2y ago

> There is research that shows exposure to suicide and self injury content increases suicidal ideation. Yes. However, I find this obsession with harm-based value judgment to the exclusion of all other considerations ethically problematic, to put it mildly. Ethics does not reduce solely to considerations of harm.

philjohn 2y ago
Would you mind expanding on that please, what are the ethically problematic things you are trying to balance against this?
- naasking 2y ago
  I reject the implicit assumption that Zuckerberg has an ethical duty to reduce indirect harm from hosting content, and I also reject the implicit assumption that ethical calculations ("balancing" harm) are a meaningful way to reason about ethics. I think both of these assumptions are ethically problematic. The only reason Zuckerberg apologized to parents is because other people would have punished him had he not, which does not imply sound ethics or that he agrees with what he said. I also think normalizing an infrastructure whose sole purpose is to suppress speech is ethically problematic. Rights are rarely expanded once taken. You're right that people post abhorrent stuff on such sites, but I prefer approaches like filters over suppression. I also think that if reducing indirect harm is important to certain people (like reducing suicide), then there are known effective ways that don't require coercive power, like the public campaigns against smoking and MADD which have both had significant impacts.
  - philjohn 2y ago
    In what way is not recommending content, or demoting it in feeds, coercive? I'm sorry, but maximalist free speech positions fail in the face of reality.
- crindy 2y ago
  Freedom of expression comes to mind. If someone had a friend commit suicide, should they not be able to discuss their experience in public?

Dig1t 2y ago

“Think of the children” isn’t really a good argument for censoring completely legal political discourse, which is what has been happening. They are admitting that there has been a global push against free speech on these platforms. >There is research that shows exposure to suicide and self injury content increases suicidal ideation. I mean do you really need research to show this link? Of course it does. We are okay with slapping an “R” rating on movies and allowing parents to be the ones who decide what content their kids can see. Why can’t we decide that parents also need to be the ones to stop their kids from consuming bad content on social media?

philjohn 2y ago
Automatically demoting, not recommending and adding "mark as disturbing" screens is what's going away - which is akin to the "R" rating. But at this point, I'm siding with the "no social media for adolescents" people more and more.

jarsin 2y ago

I've already seen disturbing stuff on X since Elon took over that I never would have seen when it was twitter. They don't even show the warning "this might be harmful content" on images and videos anymore. The X algo seems to go haywire every couple of days and dumps a bunch of this crap in my feed until I block 20+ bluecheck accounts showing this crap. I believe it's only going to get worse going forward as they all adopt these policies.

jeromegv 2y ago
It’s okay to leave X.
biosboiii 2y ago
My profile is largely unused, I follow no one, and like 1/3 times I open up the front page I get straight holocaust denial threads suggested. Completely insane.

dado3212 2y ago

> So, we’re going to continue to focus these systems on tackling illegal and high-severity violations, like terrorism, child sexual exploitation, drugs, fraud and scams. I don't think this is exhaustive, and I think SSI (suicide/self-injury) + ED/etc. stuff is considered high-severity.

philjohn 2y ago
Fingers crossed.

zahlman 2y ago

Counterpoint: censorship inherently harms everyone. People I follow on Youtube have repeatedly had their ability to discuss topics such as suicide seriously interfered with. It actually gets in the way of factual reporting when a suicide occurs in the community and of discussing the facts of the situation so that people can learn from it and possibly prevent future deaths. Not to mention, people just straight up have a right to talk about these things. It is not moral to hold one person responsible for an unintended and not reasonably foreseeable reaction to the discussion. And joking about these topics is legitimately therapeutic for some.

philjohn 2y ago
I'm not talking about that here - and that always fell under protecting voice - if mistakes were made they should have been reversed on appeal. e.g. imagery of healed scars in the context of recovery, discussions of struggles with mental health, suicidal ideation etc. I'm talking about graphic images of self harm, suicide, eating disorders. And at some point you have to weigh the maximalist interpretation of free speech "you have to host whatever I want, as long as it's not illegal" with "promoting this stuff causes active harm, no".
- zahlman 2y ago
  >And at some point you have to weigh the maximalist interpretation of free speech "you have to host whatever I want, as long as it's not illegal" with "promoting this stuff causes active harm, no". The burden of proof is on you to demonstrate that it causes such harm. I don't generally think people should be held responsible for the unintended reaction to their speech of a small minority of the audience.
  - philjohn 2y ago
    Having a piece of content removed, or demoted and not recommended, is being held responsible? Also as per the inquest into the death of Molly Russel found based on the preponderance of evidence, exposure to this kind of graphic content was largely the causative agent in her suicide. What would the bar you require be, is there a bar?