9 ms·

Exposure to many viewpoints, including wrong ones, provides a counterbalancing effect. When you actively try and suppress information you create a “forbidden k

Exposure to many viewpoints, including wrong ones, provides a counterbalancing effect. When you actively try and suppress information you create a “forbidden knowledge” effect where people seek out silos where extreme and wrongheaded information gets passed without the “sunlight is the best disinfectant”—-it grows faster…becomes more wrong, more extreme, and more dangerous. Seems to me in my experience after decades of watching and participating in online discussion extremism really only became more problematic when fact checking and active efforts to suppress took hold. Whatever the good intentions may have been, the results were worse.

ceejayoz 2y ago
There's some academic research to the contrary; banning /r/fatpeoplehate and /r/coontown on Reddit reduced incidents of hateful speech across the platform. https://www.reddit.com/r/science/comments/6zg6w6/reddits_bans_of_rcoontown_and_rfatpeoplehate/ https://www.reddit.com/r/science/comments/6zg6w6/reddits_ban... / https://comp.social.gatech.edu/papers/cscw18-chand-hate.pdf https://comp.social.gatech.edu/papers/cscw18-chand-hate.pdf "Sunlight is the best disinfectant" is a great pithy slogan, but modern society needs bleach and chlorhexidine sometimes.
- naasking 2y ago
  > There's some academic research to the contrary; banning /r/fatpeoplehate and /r/coontown on Reddit reduced incidents of hateful speech across the platform. That does not imply it reduced hateful speech overall, maybe the censorship just increased antipathy and drove that speech underground or to other platforms where it couldn't be seen.
  - ceejayoz 2y ago
    "Off Reddit" is a win. Recruitment in neutral-ish venues like Reddit is critical for extremist groups; people aren't starting on Stormfront.
    - naasking 2y ago
      That's still just a conjecture of a meaningful effect. Recruiters are able to change tactics in response you know. You're just naively assuming that those old tactics worked better just because reddit itself changed, but it could very well be the case that the more extreme rhetoric only attracted people who were already extremist and turned off moderates, but a more moderate approach that's now required could funnel more moderate people into an extremist pipeline. "Off reddit" is just a win for reddit's PR, and that's why they did it, and no other reason and no other effects can be inferred.
      - intended 2y ago
        The claim you are addressing is a separate one from the fatpeople hate story. And that claim is evidenced, It’s not conjecture. I dont have it handy on me, but we have mapped out the ways people are recruited, and things like fatpeoplehate, coontown, are the funnels for groups to find new recruits. Here’s one - https://dl.acm.org/doi/abs/10.1145/3447535.3462504 https://dl.acm.org/doi/abs/10.1145/3447535.3462504 There’s several others on things from ISIS to hacktivists. The mechanism is the same, heck - “red pill” is the term for this, it’s actually quite known.
    - Karrot_Kream 2y ago
      Not necessarily. If it drives the content off Reddit but onto another platform that's friendly to only these extremists and their views then you may just end up radicalizing the members of the original banned subs even more. I don't know if that's what happened and there's probably a lot more research to do here but I'm not convinced that deplatforming is actually a good outcome societally without more data.
- miramba 2y ago
  [flagged]
  - ceejayoz 2y ago
    Against what, microbes? There’s a reason surgeons disinfect their hands with more than a skylight. Sunlight is a shitty disinfectant.
    - miramba 2y ago
      I wasn‘t aware that society means surgery. Likewise that veiled means literal. By extension, ethnic cleansing probably means giving certain parts of a population a well deserved bath? Edit: I did not want to imply that you meant it that way. But in a different context, or coming from the wrong person, it may sound like a dog whistle.
- kcplate 2y ago
  So your example is two places that were intentionally moderated to be hateful and also suppressed the non-hateful speech in those subreddits? So removing a censored platform eliminated the problem? Amazing how that works!
  - Barrin92 2y ago
    No, you should actually go and read the paper. It didn't just reduce the type of content posted in the subreddit, they tracked individual users that were active and their behavior overall changed, including in other subreddits compared to before. Essentially what it showed was that if you pull people out of a particular echo chamber, then that had a sustained effect on how they behaved. Which is evidence contrary to the often made claim that they'd just leave and go somewhere else. It's in line with the theory that the internet fosters extremism because it enables insular pathological communities that in the analog era you'd have been slapped out of long ago by people who aren't nuts.
    - kcplate 2y ago
      > Essentially what it showed was that if you pull people out of a particular echo chamber, then that had a sustained effect on how they behaved. So…silos and echo chambers are bad. Seems to me that was part of my original point. I am suggesting that censorship of information leads people to the silos.
      - intended 2y ago
        So you are saying, that things got better when people were banned. Because when they got banned, many other communities saw improvements as well, not just those?
        kcplate 2y ago
        No I am saying that when you censor/suppress debate in the public square you drive people underground where they land in echo chambers and develop extreme views because they don’t have public debate. You don’t need to ban people from echo chambers if they don’t land there in the first place. Your solution is reactive to a problem you caused. My solution is don’t create the problem in the first place.
- raxxor 2y ago
  Maybe it reduced hate on this single metric, but the complaint is more about the errors in fact checking. And single subreddits aren't really convincing about the reliability of fact checkers if their independence is in question. In the end they do rely on a truth-authority, which is problematic, especially for political content. And Meta reported that political demands increased.
notahacker 2y ago
> Seems to me in my experience after decades of watching and participating in online discussion extremism really only became more problematic when fact checking and active efforts to suppress took hold. Whatever the good intentions may have been, the results were worse. Seems like the opposite. Traditionally we only had siloed forums which were often heavily moderated by volunteers who considered the forums their personal fiefdom, read every single thread and deleted stuff for being "off topic" never mind objectionable, plus the odd place like /b/ which revelled in being unmoderated. Then you ended up with more people on big platforms that were comparatively-speaking, pretty lightly and reactively moderated. Then you ended up with politicians weighing in against moderation with the suggestion even annotating content published on their platform was a free speech violation, let alone refraining from continuing to publish it. The difference between antivax sentiment now and circa 2005 isn't that nobody ever determined that they weren't having that nonsense on their forums or closed threads with links to Snopes back then or that it's become difficult to find any references to it outside antivaxxer communities since then. Quite the opposite, the difference is that it's now coming from the mouth of a presumptive Health Secretary, amplified on allied news networks and now we have corporations running scared that labelling it a hoax might run the risk of offending the people in charge. Turns out sunlight is a catalyst for growth
- kcplate 2y ago
  > The difference between antivax sentiment now and circa 2005 The antivax movement literally grew exponentially when vaccine information started to be actively censored on the largest social media platforms and you think that is because there wasn’t enough censorship? People were literally driven into antivax information silos because a bunch of idiots decided that vaccine criticism should be forbidden in the public square Wow.
  - notahacker 2y ago
    Sorry, but I live in a country using exactly the same social media providers as you, subject to exactly the same (actually pretty limited) censorship and without widespread, committed and politically-aligned antivax sentiment People in the US didn't need to be "driven into antivax information silos", because those antivax information silos were their favourite talk show hosts and some of the country's most prominent politicians. Turns out that promotion of antivax sentiment as an important issue that must be discussed and constant attacks on public health officials doesn't "disinfect" people against the belief that there might be some truth to it...
hmmm-i-wonder 2y ago
>Exposure to many viewpoints, including wrong ones, provides a counterbalancing effect. When you actively try and suppress information you create a “forbidden knowledge” effect where people seek out silos where extreme and wrongheaded information gets passed without the “sunlight is the best disinfectant”—-it grows faster…becomes more wrong, more extreme, and more dangerous. Fact checkers don't suppress information, they add context and information to posts others make and provide the exposure to many viewpoints that echo chambers often do not have. People haven't stopped posting wrong and biased information with fact checkers, they just have the counterpoint to their bullshit displayed alongside their posts on the platform. >Seems to me in my experience after decades of watching and participating in online discussion extremism really only became more problematic when fact checking and active efforts to suppress took hold. Whatever the good intentions may have been, the results were worse. My decades of watching is exactly the opposite. Extremism is and was rampant long before fact checking, and fact checking really only served to push some of the most extreme content to the margins and to smaller platforms that don't have it. It concentrates it in some ways as many of these opinions fall apart quickly when exposed to truth and facts.
- freedomben 2y ago
  > Fact checkers don't suppress information, I think some moderation is important, but misrepresenting fact checkers (damn ironic actually) doesn't serve us. Of course fact check suppresses information! That's the whole point. Sometimes it results in straight up deletion, but even when not it results in lowered reach aka suppression of what the algorithm would normally allow to trend, etc.
  - hmmm-i-wonder 2y ago
    >Of course fact check suppresses information! That's the whole point Its not. The fact checkers in this case, and almost all cases we're discussing ADD information that challenges the posted data, not censor or restrict it from being posted. Outside of illegal content that is. Content deemed illegal was removed by moderation teams, this was before fact checking, and will continue with community notes with little to no change.
- 2y ago
fzeroracer 2y ago
> Seems to me in my experience after decades of watching and participating in online discussion extremism really only became more problematic when fact checking and active efforts to suppress took hold. Whatever the good intentions may have been, the results were worse. This is just overtly and flatly wrong. I reject your experience fully because over the past few decades the internet has become more open, not less. We openly debated people that believed vaccines caused autism and gave them microphones. Every single loud asshole and dipshit was given maximum volume on whatever radio show or podcast or social media platform they could want.
- kcplate 2y ago
  You can reject my experience all you want but the reality is that between 2020 and 2023ish the world’s top social media platforms became less open about specific kinds of information and actively tried to censor and suppress any contrary information to a government opinion/narrative about certain subjects. During this time certain forms of extremism exploded in popularity as people were driven to information silos to find and learn about the information that the social media platforms were trying to suppress. Those silos generally didn’t have censorship but they also didn’t have contrarian voices either. So when folks landed in those silos all they heard was the assholes at the loud volumes and without the contrarians, followed those assholes. Specifically to vaccines, the antivax crowd was pretty minimal to a some nutjob soccer moms, holistic medicine fanatics, and RFKjr until you stopped having conversations with them, because you folks who want or believe that censorship is good silenced the debate and did not follow them to the forums where they went to spread their ideas to continue the debate. I am absolutely convinced that the growth in the antivax movement is directly tied to the censorship effort (and the desire of the government to not be completely honest about the vaccines at the time).
  - jasdi 2y ago
    No free lunch here. Social media is different from systems in the past cuz it give everyone Free Broadcast capability. In the past people were told they had Free Speech, but they didn't have Free access to Broadcast Media (newspapers/radio/tv/movie studios/satellites). It was always up to someone else with Access to Broadcast(one to all messaging) to prop up voices they thought was important. Shannon's Information theory tells us Social Media as a system can't work cause - once you tell people their voice matters, give everyone in the room a mic, plugged into the same sound system, and allow everyone to speak, firstly you get massive noise, secondly as a reaction people will scream louder and louder and repeat their message more and more. Noise only compounds. The math says it can't work. The way people are debating about this is under an assumption that it can.
thaw24612107 2y ago
[dead]
intended 2y ago
The principle is sound, but it’s a principle. The mechanisms of online speech show us a few other issues. For example certain ideas are far more “fit” for transmission and memory than others. Take a look at something as commonplace as “ghosts” or the idea of penguins. Ghosts are in all cultures, and they are essentially people with some additional properties. Penguins are birds that dont fly. Brains absorb stories and ideas like flightless birds easily, because they build on pre existing concepts. Talk about spacetime, or multiple dimensions and you aren’t going to have the same degree of uptake. So when I put certain ideas into competition with each other, all else being equal - the more suited for human foibles, the more successful the idea. People also dont make that much effort to seek out forbidden knowledge. Conservative main stream media has made many things forbidden - 1/3rd of America isnt aware that Obamacare and the ACA are the same thing. Sunlight is the best disinfectant for certain breeds of germs. Many others get on just fine. In my many decades of online existence, which includes being on multiple sides of moderation, extremism was on the rise from before, because we had created the arguments and structures that thrive on it. Content moderation was a hap hazard effort created out of necessity to stall it. Personally - I hope this works. Moderation sucks, and is straight up traumatic. If we can get better, more effective market places of ideas, then I am all for it. I care about the effectiveness of the exchange of ideas. I see free speech as a principle that supports this. But the goal is always the functioning of the marketplace.

ceejayoz 2y ago

There's some academic research to the contrary; banning /r/fatpeoplehate and /r/coontown on Reddit reduced incidents of hateful speech across the platform. https://www.reddit.com/r/science/comments/6zg6w6/reddits_bans_of_rcoontown_and_rfatpeoplehate/ https://www.reddit.com/r/science/comments/6zg6w6/reddits_ban... / https://comp.social.gatech.edu/papers/cscw18-chand-hate.pdf https://comp.social.gatech.edu/papers/cscw18-chand-hate.pdf "Sunlight is the best disinfectant" is a great pithy slogan, but modern society needs bleach and chlorhexidine sometimes.

naasking 2y ago
> There's some academic research to the contrary; banning /r/fatpeoplehate and /r/coontown on Reddit reduced incidents of hateful speech across the platform. That does not imply it reduced hateful speech overall, maybe the censorship just increased antipathy and drove that speech underground or to other platforms where it couldn't be seen.
- ceejayoz 2y ago
  "Off Reddit" is a win. Recruitment in neutral-ish venues like Reddit is critical for extremist groups; people aren't starting on Stormfront.
  - naasking 2y ago
    That's still just a conjecture of a meaningful effect. Recruiters are able to change tactics in response you know. You're just naively assuming that those old tactics worked better just because reddit itself changed, but it could very well be the case that the more extreme rhetoric only attracted people who were already extremist and turned off moderates, but a more moderate approach that's now required could funnel more moderate people into an extremist pipeline. "Off reddit" is just a win for reddit's PR, and that's why they did it, and no other reason and no other effects can be inferred.
    - intended 2y ago
      The claim you are addressing is a separate one from the fatpeople hate story. And that claim is evidenced, It’s not conjecture. I dont have it handy on me, but we have mapped out the ways people are recruited, and things like fatpeoplehate, coontown, are the funnels for groups to find new recruits. Here’s one - https://dl.acm.org/doi/abs/10.1145/3447535.3462504 https://dl.acm.org/doi/abs/10.1145/3447535.3462504 There’s several others on things from ISIS to hacktivists. The mechanism is the same, heck - “red pill” is the term for this, it’s actually quite known.
  - Karrot_Kream 2y ago
    Not necessarily. If it drives the content off Reddit but onto another platform that's friendly to only these extremists and their views then you may just end up radicalizing the members of the original banned subs even more. I don't know if that's what happened and there's probably a lot more research to do here but I'm not convinced that deplatforming is actually a good outcome societally without more data.
miramba 2y ago
[flagged]
- ceejayoz 2y ago
  Against what, microbes? There’s a reason surgeons disinfect their hands with more than a skylight. Sunlight is a shitty disinfectant.
  - miramba 2y ago
    I wasn‘t aware that society means surgery. Likewise that veiled means literal. By extension, ethnic cleansing probably means giving certain parts of a population a well deserved bath? Edit: I did not want to imply that you meant it that way. But in a different context, or coming from the wrong person, it may sound like a dog whistle.
kcplate 2y ago
So your example is two places that were intentionally moderated to be hateful and also suppressed the non-hateful speech in those subreddits? So removing a censored platform eliminated the problem? Amazing how that works!
- Barrin92 2y ago
  No, you should actually go and read the paper. It didn't just reduce the type of content posted in the subreddit, they tracked individual users that were active and their behavior overall changed, including in other subreddits compared to before. Essentially what it showed was that if you pull people out of a particular echo chamber, then that had a sustained effect on how they behaved. Which is evidence contrary to the often made claim that they'd just leave and go somewhere else. It's in line with the theory that the internet fosters extremism because it enables insular pathological communities that in the analog era you'd have been slapped out of long ago by people who aren't nuts.
  - kcplate 2y ago
    > Essentially what it showed was that if you pull people out of a particular echo chamber, then that had a sustained effect on how they behaved. So…silos and echo chambers are bad. Seems to me that was part of my original point. I am suggesting that censorship of information leads people to the silos.
    - intended 2y ago
      So you are saying, that things got better when people were banned. Because when they got banned, many other communities saw improvements as well, not just those?
      - kcplate 2y ago
        No I am saying that when you censor/suppress debate in the public square you drive people underground where they land in echo chambers and develop extreme views because they don’t have public debate. You don’t need to ban people from echo chambers if they don’t land there in the first place. Your solution is reactive to a problem you caused. My solution is don’t create the problem in the first place.
raxxor 2y ago
Maybe it reduced hate on this single metric, but the complaint is more about the errors in fact checking. And single subreddits aren't really convincing about the reliability of fact checkers if their independence is in question. In the end they do rely on a truth-authority, which is problematic, especially for political content. And Meta reported that political demands increased.

notahacker 2y ago

> Seems to me in my experience after decades of watching and participating in online discussion extremism really only became more problematic when fact checking and active efforts to suppress took hold. Whatever the good intentions may have been, the results were worse. Seems like the opposite. Traditionally we only had siloed forums which were often heavily moderated by volunteers who considered the forums their personal fiefdom, read every single thread and deleted stuff for being "off topic" never mind objectionable, plus the odd place like /b/ which revelled in being unmoderated. Then you ended up with more people on big platforms that were comparatively-speaking, pretty lightly and reactively moderated. Then you ended up with politicians weighing in against moderation with the suggestion even annotating content published on their platform was a free speech violation, let alone refraining from continuing to publish it. The difference between antivax sentiment now and circa 2005 isn't that nobody ever determined that they weren't having that nonsense on their forums or closed threads with links to Snopes back then or that it's become difficult to find any references to it outside antivaxxer communities since then. Quite the opposite, the difference is that it's now coming from the mouth of a presumptive Health Secretary, amplified on allied news networks and now we have corporations running scared that labelling it a hoax might run the risk of offending the people in charge. Turns out sunlight is a catalyst for growth

kcplate 2y ago
> The difference between antivax sentiment now and circa 2005 The antivax movement literally grew exponentially when vaccine information started to be actively censored on the largest social media platforms and you think that is because there wasn’t enough censorship? People were literally driven into antivax information silos because a bunch of idiots decided that vaccine criticism should be forbidden in the public square Wow.
- notahacker 2y ago
  Sorry, but I live in a country using exactly the same social media providers as you, subject to exactly the same (actually pretty limited) censorship and without widespread, committed and politically-aligned antivax sentiment People in the US didn't need to be "driven into antivax information silos", because those antivax information silos were their favourite talk show hosts and some of the country's most prominent politicians. Turns out that promotion of antivax sentiment as an important issue that must be discussed and constant attacks on public health officials doesn't "disinfect" people against the belief that there might be some truth to it...

hmmm-i-wonder 2y ago

>Exposure to many viewpoints, including wrong ones, provides a counterbalancing effect. When you actively try and suppress information you create a “forbidden knowledge” effect where people seek out silos where extreme and wrongheaded information gets passed without the “sunlight is the best disinfectant”—-it grows faster…becomes more wrong, more extreme, and more dangerous. Fact checkers don't suppress information, they add context and information to posts others make and provide the exposure to many viewpoints that echo chambers often do not have. People haven't stopped posting wrong and biased information with fact checkers, they just have the counterpoint to their bullshit displayed alongside their posts on the platform. >Seems to me in my experience after decades of watching and participating in online discussion extremism really only became more problematic when fact checking and active efforts to suppress took hold. Whatever the good intentions may have been, the results were worse. My decades of watching is exactly the opposite. Extremism is and was rampant long before fact checking, and fact checking really only served to push some of the most extreme content to the margins and to smaller platforms that don't have it. It concentrates it in some ways as many of these opinions fall apart quickly when exposed to truth and facts.

freedomben 2y ago
> Fact checkers don't suppress information, I think some moderation is important, but misrepresenting fact checkers (damn ironic actually) doesn't serve us. Of course fact check suppresses information! That's the whole point. Sometimes it results in straight up deletion, but even when not it results in lowered reach aka suppression of what the algorithm would normally allow to trend, etc.
- hmmm-i-wonder 2y ago
  >Of course fact check suppresses information! That's the whole point Its not. The fact checkers in this case, and almost all cases we're discussing ADD information that challenges the posted data, not censor or restrict it from being posted. Outside of illegal content that is. Content deemed illegal was removed by moderation teams, this was before fact checking, and will continue with community notes with little to no change.
2y ago

fzeroracer 2y ago

kcplate 2y ago
You can reject my experience all you want but the reality is that between 2020 and 2023ish the world’s top social media platforms became less open about specific kinds of information and actively tried to censor and suppress any contrary information to a government opinion/narrative about certain subjects. During this time certain forms of extremism exploded in popularity as people were driven to information silos to find and learn about the information that the social media platforms were trying to suppress. Those silos generally didn’t have censorship but they also didn’t have contrarian voices either. So when folks landed in those silos all they heard was the assholes at the loud volumes and without the contrarians, followed those assholes. Specifically to vaccines, the antivax crowd was pretty minimal to a some nutjob soccer moms, holistic medicine fanatics, and RFKjr until you stopped having conversations with them, because you folks who want or believe that censorship is good silenced the debate and did not follow them to the forums where they went to spread their ideas to continue the debate. I am absolutely convinced that the growth in the antivax movement is directly tied to the censorship effort (and the desire of the government to not be completely honest about the vaccines at the time).
- jasdi 2y ago
  No free lunch here. Social media is different from systems in the past cuz it give everyone Free Broadcast capability. In the past people were told they had Free Speech, but they didn't have Free access to Broadcast Media (newspapers/radio/tv/movie studios/satellites). It was always up to someone else with Access to Broadcast(one to all messaging) to prop up voices they thought was important. Shannon's Information theory tells us Social Media as a system can't work cause - once you tell people their voice matters, give everyone in the room a mic, plugged into the same sound system, and allow everyone to speak, firstly you get massive noise, secondly as a reaction people will scream louder and louder and repeat their message more and more. Noise only compounds. The math says it can't work. The way people are debating about this is under an assumption that it can.

thaw24612107 2y ago

[dead]

intended 2y ago

The principle is sound, but it’s a principle. The mechanisms of online speech show us a few other issues. For example certain ideas are far more “fit” for transmission and memory than others. Take a look at something as commonplace as “ghosts” or the idea of penguins. Ghosts are in all cultures, and they are essentially people with some additional properties. Penguins are birds that dont fly. Brains absorb stories and ideas like flightless birds easily, because they build on pre existing concepts. Talk about spacetime, or multiple dimensions and you aren’t going to have the same degree of uptake. So when I put certain ideas into competition with each other, all else being equal - the more suited for human foibles, the more successful the idea. People also dont make that much effort to seek out forbidden knowledge. Conservative main stream media has made many things forbidden - 1/3rd of America isnt aware that Obamacare and the ACA are the same thing. Sunlight is the best disinfectant for certain breeds of germs. Many others get on just fine. In my many decades of online existence, which includes being on multiple sides of moderation, extremism was on the rise from before, because we had created the arguments and structures that thrive on it. Content moderation was a hap hazard effort created out of necessity to stall it. Personally - I hope this works. Moderation sucks, and is straight up traumatic. If we can get better, more effective market places of ideas, then I am all for it. I care about the effectiveness of the exchange of ideas. I see free speech as a principle that supports this. But the goal is always the functioning of the marketplace.