4 ms·

This exactly mirrors my thoughts, although I don't work in your field. One quote: "For example, in December 2024, we removed millions of pieces of content ever

This exactly mirrors my thoughts, although I don't work in your field. One quote: "For example, in December 2024, we removed millions of pieces of content every day. While these actions account for less than 1% of content produced every day, we think one to two out of every 10 of these actions may have been mistakes (i.e., the content may not have actually violated our policies)." That is first order data and it's interesting. However, before making policy decisions, I would want the second order data: what is the human cost of those mistakes, and what percentage of policy-violating content will not be removed as a result of these changes? Finally, what's the cost of not removing that percentage? For that matter, by talking about the percentage of active mistakes without saying how many policy violations are currently missed, you're framing the debate in a certain direction.

philjohn 2y ago
Indeed. The human cost of a piece of content being taken down depends on the piece of content, and the reason behind posting it. In the case of someone posting about recovery from self injury and including a photo of their healed self-harm scars, having that taken down by mistake would be more harmful than someone who posted a cartoon depiction of suicide for the lolz.
- BryantD 2y ago
  Yes. My personal belief, for whatever that's worth, is that communication and speech are one of the most powerful tools any of us have. Talking can change minds, move societies, arouse emotions, and in general makes a difference. This is true no matter the format (text, voice, etc.). That means that restricting communication should not be a casual activity. Free speech is a good ideal for a reason. It also means that, if you believe in the primacy of free speech, you are obligated to consider the implications of that belief. Speech has effects. In my adult life, since 1990, we have seen a major change in the ease of communication. IMHO, society hasn't been able to fully adjust to that change -- or rather, that huge suite of changes. I sincerely do not know what a healthy society using the Internet looks like; I don't think we're in one now. All of these arguments (on all sides, mine included) are hampered by our lack of perspective.
  - philjohn 2y ago
    Which is why we should research this carefully - and the research thus far points to consumption or graphic or even borderline depictions of suicide, self injury and eating disorder content (eg thinspo) being bad for mental health in at least teens. Meta seem to be making the case for those who would see social media banned for people under the age of 18. To enforce that properly would require needing ID, and that then opens a whole can of civil liberty issues.
    - nradov 2y ago
      The social "science" research in this area is junk with small effect sizes, unclear causality, and multiple uncontrolled variables. People who claim to be following the science in this area are generally being disingenuous and picking results that support their preferred ideology.
      - philjohn 2y ago
        The ideology of ... not doing something that could make adolescent (and adult) mental health worse, to the point of suicide? Yeah, making that my ideology is a hill I'm willing to die on, sorry.
        CamperBob2 2y ago
        Forcing the entire world to conform to your idea of "child-safe" has negative consequences, too.
        philjohn 2y ago
        Can you share the negative consequences of not allowing, and not promoting, graphic images of self harm and suicide on a social media network please?
        CamperBob2 2y ago
        It gives a lot of unearned power to those who decide what constitutes "promoting," "graphic," "self harm," and "social media," for one thing. If you or I happen to agree with the people who wield that power, rest assured it's only a temporary coincidence.
- ben_w 2y ago
  Given how easy it is to take things out of context, I'm not so sure that the original context really makes a difference. There's more people online than any of us has heartbeats, and the n^2 number of user-user pairs generates detrimental effects that track any positive effects. Much better, I think, for each of us to have a small and private personal social network, not to hand everything over to a foreign* company trying to project its social norms worldwide. * Facebook claims about 3 billion active users, so for 89%-93.5%** of its users, the fact that Facebook is American makes them foreign. ** https://thesocialshepherd.com/blog/facebook-statistics#:~:text=The%20U.S.%20population%20currently%20sits,States%20were%20regular%20Facebook%20users https://thesocialshepherd.com/blog/facebook-statistics#:~:te....
barbazoo 2y ago
> we think one to two out of every 10 of these actions may have been mistakes May have been a mistake? Reminds me of RTO and the subjective feeling of being more productive in the office. They have the feeling they may have made mistakes and base their new policy on that feeling.
- theptip 2y ago
  I think what they are saying there is the press release interpretation of experiments showing a false positive rate of 10-20%, with error bars wide enough that stating a percentage gives too many significant figures. But the definition of FP is necessarily fuzzy; if you can perfectly identify them as FP at scale then you have built a better classifier and you no longer have the FP problem. So any statement about FP rates necessarily needs to be couched in uncertainty. I don't think it's malicious wordsmithing where they are mis-representing the internal data, though I don't have the data to confirm.
nradov 2y ago
The human cost can't be quantified in any meaningfully precise way on either side. The calculations are necessarily based on so many assumptions as to become entirely subjective. Ultimately the decisions will be made based on politics and business priorities, not any objective calculation of human cost.
theptip 2y ago
> However, before making policy decisions, I would want the second order data: I think this the wrong lens. The correct lens is: if they don't voluntarily make this change, will they be forced to? The incoming administration seems committed to banning "censorship", so I believe making a cost/benefit analysis is something of a false choice. E.g. see https://www.youtube.com/watch?v=xJfUXVOoFBo https://www.youtube.com/watch?v=xJfUXVOoFBo
- philjohn 2y ago
  That ignores the regulations in the EU, and the UK (coming into force this year), and also the huge volume of lawsuits they are facing in the US. Does everyone remember Zuck turning around to apologise to the parents in that senate hearing? Those parents must feel this is a slap in the face.
  - theptip 2y ago
    This is a decision for the US market first and foremost. The lawsuits you mention are sadly irrelevant to the decision-making; again, if you are about to be forced to make this change by Trump, the results of some cost/benefit study will not sway his reasoning. His decision is already made. FWIW I would not be surprised if the bluster about championing free speech abroad gets quietly forgotten; we’ll see. They explicitly state they will comply with laws, which in EU likely means continuing to moderate (more not less over time, given the regulatory trends).
    - philjohn 2y ago
      I mean, we can tell it's not about free speech because it specifically allows you to target gay and trans people, calling them mentally ill, but not religious people.

philjohn 2y ago

Indeed. The human cost of a piece of content being taken down depends on the piece of content, and the reason behind posting it. In the case of someone posting about recovery from self injury and including a photo of their healed self-harm scars, having that taken down by mistake would be more harmful than someone who posted a cartoon depiction of suicide for the lolz.

BryantD 2y ago
Yes. My personal belief, for whatever that's worth, is that communication and speech are one of the most powerful tools any of us have. Talking can change minds, move societies, arouse emotions, and in general makes a difference. This is true no matter the format (text, voice, etc.). That means that restricting communication should not be a casual activity. Free speech is a good ideal for a reason. It also means that, if you believe in the primacy of free speech, you are obligated to consider the implications of that belief. Speech has effects. In my adult life, since 1990, we have seen a major change in the ease of communication. IMHO, society hasn't been able to fully adjust to that change -- or rather, that huge suite of changes. I sincerely do not know what a healthy society using the Internet looks like; I don't think we're in one now. All of these arguments (on all sides, mine included) are hampered by our lack of perspective.
- philjohn 2y ago
  Which is why we should research this carefully - and the research thus far points to consumption or graphic or even borderline depictions of suicide, self injury and eating disorder content (eg thinspo) being bad for mental health in at least teens. Meta seem to be making the case for those who would see social media banned for people under the age of 18. To enforce that properly would require needing ID, and that then opens a whole can of civil liberty issues.
  - nradov 2y ago
    The social "science" research in this area is junk with small effect sizes, unclear causality, and multiple uncontrolled variables. People who claim to be following the science in this area are generally being disingenuous and picking results that support their preferred ideology.
    - philjohn 2y ago
      The ideology of ... not doing something that could make adolescent (and adult) mental health worse, to the point of suicide? Yeah, making that my ideology is a hill I'm willing to die on, sorry.
      - CamperBob2 2y ago
        Forcing the entire world to conform to your idea of "child-safe" has negative consequences, too.
        philjohn 2y ago
        Can you share the negative consequences of not allowing, and not promoting, graphic images of self harm and suicide on a social media network please?
        CamperBob2 2y ago
        It gives a lot of unearned power to those who decide what constitutes "promoting," "graphic," "self harm," and "social media," for one thing. If you or I happen to agree with the people who wield that power, rest assured it's only a temporary coincidence.
ben_w 2y ago
Given how easy it is to take things out of context, I'm not so sure that the original context really makes a difference. There's more people online than any of us has heartbeats, and the n^2 number of user-user pairs generates detrimental effects that track any positive effects. Much better, I think, for each of us to have a small and private personal social network, not to hand everything over to a foreign* company trying to project its social norms worldwide. * Facebook claims about 3 billion active users, so for 89%-93.5%** of its users, the fact that Facebook is American makes them foreign. ** https://thesocialshepherd.com/blog/facebook-statistics#:~:text=The%20U.S.%20population%20currently%20sits,States%20were%20regular%20Facebook%20users https://thesocialshepherd.com/blog/facebook-statistics#:~:te....

barbazoo 2y ago

> we think one to two out of every 10 of these actions may have been mistakes May have been a mistake? Reminds me of RTO and the subjective feeling of being more productive in the office. They have the feeling they may have made mistakes and base their new policy on that feeling.

theptip 2y ago
I think what they are saying there is the press release interpretation of experiments showing a false positive rate of 10-20%, with error bars wide enough that stating a percentage gives too many significant figures. But the definition of FP is necessarily fuzzy; if you can perfectly identify them as FP at scale then you have built a better classifier and you no longer have the FP problem. So any statement about FP rates necessarily needs to be couched in uncertainty. I don't think it's malicious wordsmithing where they are mis-representing the internal data, though I don't have the data to confirm.

nradov 2y ago

The human cost can't be quantified in any meaningfully precise way on either side. The calculations are necessarily based on so many assumptions as to become entirely subjective. Ultimately the decisions will be made based on politics and business priorities, not any objective calculation of human cost.

theptip 2y ago

> However, before making policy decisions, I would want the second order data: I think this the wrong lens. The correct lens is: if they don't voluntarily make this change, will they be forced to? The incoming administration seems committed to banning "censorship", so I believe making a cost/benefit analysis is something of a false choice. E.g. see https://www.youtube.com/watch?v=xJfUXVOoFBo https://www.youtube.com/watch?v=xJfUXVOoFBo

philjohn 2y ago
That ignores the regulations in the EU, and the UK (coming into force this year), and also the huge volume of lawsuits they are facing in the US. Does everyone remember Zuck turning around to apologise to the parents in that senate hearing? Those parents must feel this is a slap in the face.
- theptip 2y ago
  This is a decision for the US market first and foremost. The lawsuits you mention are sadly irrelevant to the decision-making; again, if you are about to be forced to make this change by Trump, the results of some cost/benefit study will not sway his reasoning. His decision is already made. FWIW I would not be surprised if the bluster about championing free speech abroad gets quietly forgotten; we’ll see. They explicitly state they will comply with laws, which in EU likely means continuing to moderate (more not less over time, given the regulatory trends).
  - philjohn 2y ago
    I mean, we can tell it's not about free speech because it specifically allows you to target gay and trans people, calling them mentally ill, but not religious people.