6 ms·

This part is scary: The moderators told me it’s a place where the conspiracy videos and memes that they see each day gradually lead them to embrace fringe view

This part is scary: The moderators told me it’s a place where the conspiracy videos and memes that they see each day gradually lead them to embrace fringe views. One auditor walks the floor promoting the idea that the Earth is flat. A former employee told me he has begun to question certain aspects of the Holocaust. Another former employee, who told me he has mapped every escape route out of his house and sleeps with a gun at his side, said: “I no longer believe 9/11 was a terrorist attack.” I have a hope that people will eventually develop immunity to conspiracy theories after falling for enough of them and eventually seeing them disproved. But with things like pizzagate/Qanon it seems some people just keep going deeper.

monetus 8y ago
When it comes to neuroplasticity and change, all it takes is time. If you spend more in the shoes of those flagged for moderation than their opposites, maybe its inevitable that you attain a form of Stockholm syndrome.
- megous 8y ago
  It's a bit cultlike. Also add isolation from family/friends, and tight control of what you can and can't do, and having no trusted person (therapist there works for the comapny). It doesn't surprise me that it can lead to similar outcomes, as being in a cult.
rgoulter 8y ago
To extend the metaphor "develop immunity" a bit, I'd be curious about the mappings to: - vaccines, which build immunity by providing an inert or weak threat that the body builds resistance to. - herd immunity: restricts the spread of diseases if a sufficiently high proportion of the population is immune to the disease.
Diederich 8y ago
> seeing them disproved Well, many of these things essentially have a starting state as 'disproved', ya?
- lazugod 8y ago
  Not really? Babies aren’t born with the knowledge that certain conspiracy theories are bunk.
  - isoskeles 8y ago
    I think the intent of the comment was that some things start as conspiracy theories but end up being true, e.g. MK-ULTRA. Although what I'm curious about in my own example is, how much of a conspiracy theory was this before it was confirmed, as in, were there many people saying that the CIA is doing mind control experiments? Less important but related, how much do people distinguish between that reality and sort of 'corollary' / related conspiracy theories that are not proven (as an example, that RFK's assassinator was a subject of CIA mind control through MK-ULTRA -- this is unproven but people might assume it's true when they find out that MK-ULTRA was a real program).
jrochkind1 8y ago
This story even suggests the _opposite_ of immunity--the more you see of them, the more likely you are to buy into them. It's not quite a good analogy, but it also reminds me of how I noticed that building/construction contractors I know almost invariably believe that the risks of asbestos and lead paint are highly overrated -- I've thought it's kind of for their own cognitive self-preservation, since they unavoidably get a lot of exposure, and don't want to believe their job is killing them. But maybe it's also just got something to do with a sort of "exposure therapy" effect where whatever you are exposed to becomes normalized.
intended 8y ago
Do not hold that belief. Please realize that brains are hackable, and that there is always a set of content that will be absorbed by a brain which appears plausible to the brain, but in reality is false. Ideas can be configured to get past an audience’s mental defenses. It will happen to you, and you will see it happen to your children, your educated parents, your well mannered neighbors and more. Maybe there is a mental type which is immune to conspiracies, but that kind of immunity will come with some other mental cost. I’m seeing gentle people in my family, repeat sentences that always predicted a user was being radicalized/polarized online. The only safe move is to get off of social media.
foobiekr 8y ago
Intelligence analysts who know the material they are analyzing is propaganda still need active intervention to prevent them adopting the views they are analyzing critically. It’s just an issue with how our minds work.
faissaloo 8y ago
If you wanted to end conspiracy theories you'd have to end critical thinking.
wyldfire 8y ago
It's as if these people are cannon fodder. I never imagined that they would deserve pity, but they clearly do. I always figured the Facebooks of the world needed more moderation. But this makes me think that maybe the only effective move is to target further upstream and somehow constrain submission or account creation. I can't see how this ends any way other than with network authentication of computers with signed bootloaders that can stipulate the operator's identity, paired with criminal statutes prohibiting operating a computer under someone else's identity or defeating the authentication/encryption that makes it possible.
anigbrowl 8y ago
I have a hope that people will eventually develop immunity to conspiracy theories after falling for enough of them and eventually seeing them disproved. Conspiracy theories don't appeal to the rational. A conspiracy is built on an emotional foundation and the 'logic' is just scaffolding to hold the emotional payload. A lot of people (but (i don't know what proportion) are susceptible to emotional addiction and contagion. It's not wise or safe to assume that people can/will/should just develop cognitive resistance (though I totally understand your hope that that would happen). Sadly we have abundant and growing evidence that it is relatively easy for bad actors to cultivate antisocial behavior up to and including murderous rampages.

monetus 8y ago

When it comes to neuroplasticity and change, all it takes is time. If you spend more in the shoes of those flagged for moderation than their opposites, maybe its inevitable that you attain a form of Stockholm syndrome.

megous 8y ago
It's a bit cultlike. Also add isolation from family/friends, and tight control of what you can and can't do, and having no trusted person (therapist there works for the comapny). It doesn't surprise me that it can lead to similar outcomes, as being in a cult.

rgoulter 8y ago

To extend the metaphor "develop immunity" a bit, I'd be curious about the mappings to: - vaccines, which build immunity by providing an inert or weak threat that the body builds resistance to. - herd immunity: restricts the spread of diseases if a sufficiently high proportion of the population is immune to the disease.

Diederich 8y ago

> seeing them disproved Well, many of these things essentially have a starting state as 'disproved', ya?

lazugod 8y ago
Not really? Babies aren’t born with the knowledge that certain conspiracy theories are bunk.
- isoskeles 8y ago
  I think the intent of the comment was that some things start as conspiracy theories but end up being true, e.g. MK-ULTRA. Although what I'm curious about in my own example is, how much of a conspiracy theory was this before it was confirmed, as in, were there many people saying that the CIA is doing mind control experiments? Less important but related, how much do people distinguish between that reality and sort of 'corollary' / related conspiracy theories that are not proven (as an example, that RFK's assassinator was a subject of CIA mind control through MK-ULTRA -- this is unproven but people might assume it's true when they find out that MK-ULTRA was a real program).

jrochkind1 8y ago

This story even suggests the _opposite_ of immunity--the more you see of them, the more likely you are to buy into them. It's not quite a good analogy, but it also reminds me of how I noticed that building/construction contractors I know almost invariably believe that the risks of asbestos and lead paint are highly overrated -- I've thought it's kind of for their own cognitive self-preservation, since they unavoidably get a lot of exposure, and don't want to believe their job is killing them. But maybe it's also just got something to do with a sort of "exposure therapy" effect where whatever you are exposed to becomes normalized.

intended 8y ago

Do not hold that belief. Please realize that brains are hackable, and that there is always a set of content that will be absorbed by a brain which appears plausible to the brain, but in reality is false. Ideas can be configured to get past an audience’s mental defenses. It will happen to you, and you will see it happen to your children, your educated parents, your well mannered neighbors and more. Maybe there is a mental type which is immune to conspiracies, but that kind of immunity will come with some other mental cost. I’m seeing gentle people in my family, repeat sentences that always predicted a user was being radicalized/polarized online. The only safe move is to get off of social media.

foobiekr 8y ago

Intelligence analysts who know the material they are analyzing is propaganda still need active intervention to prevent them adopting the views they are analyzing critically. It’s just an issue with how our minds work.

faissaloo 8y ago

If you wanted to end conspiracy theories you'd have to end critical thinking.

wyldfire 8y ago

It's as if these people are cannon fodder. I never imagined that they would deserve pity, but they clearly do. I always figured the Facebooks of the world needed more moderation. But this makes me think that maybe the only effective move is to target further upstream and somehow constrain submission or account creation. I can't see how this ends any way other than with network authentication of computers with signed bootloaders that can stipulate the operator's identity, paired with criminal statutes prohibiting operating a computer under someone else's identity or defeating the authentication/encryption that makes it possible.

anigbrowl 8y ago

I have a hope that people will eventually develop immunity to conspiracy theories after falling for enough of them and eventually seeing them disproved. Conspiracy theories don't appeal to the rational. A conspiracy is built on an emotional foundation and the 'logic' is just scaffolding to hold the emotional payload. A lot of people (but (i don't know what proportion) are susceptible to emotional addiction and contagion. It's not wise or safe to assume that people can/will/should just develop cognitive resistance (though I totally understand your hope that that would happen). Sadly we have abundant and growing evidence that it is relatively easy for bad actors to cultivate antisocial behavior up to and including murderous rampages.