5 ms·

Humans aren't rational actors who get tricked into embracing propaganda by subtle logical fallacies. This will be of no more help than fact checking. It's a ne

Humans aren't rational actors who get tricked into embracing propaganda by subtle logical fallacies. This will be of no more help than fact checking. It's a neat project on its own, tbc, I just have very low expectations of broader impact.

FloorEgg 2y ago
I disagree with your first point. People are far more rational than you are making them out to be, it's just that they are rational within their own value system, not yours. Also today's propaganda is capable of adapting itself to each audience member's value system to make it more palatable, and then gradually nudge the audience towards the desired narrative/beliefs/values. The systems that distribute the propaganda are already analyzing people's values and using that information to manipulate people. I think that information asymmetry is part of the problem. I could be wrong, but I think flipping that dynamic around so the public can see the true values of the subjects of propaganda may help neutralize a lot of propaganda. As far as what impact this specific project will have, I have no idea. You may be right. I'm curious about its limitations and how it can be applied.
- kubb 2y ago
  I thought so too, but recently so many people dropped or adapted their core beliefs to be able to support and defend people in power that they really love that it made me change my mind. Now I think that value systems are malleable and are formed by whatever makes us feel good. And the logical consistency on top is very optional.
  - FloorEgg 2y ago
    Or maybe these people you speak up assume that the people in power have values aligned with their own, and if there was an unbiased system that highlights value discrepancies using formal logic, they might not "love" those people as much anymore. What I assume you might be missing is that you are looking at the world through a different lens than these other people. Both you and they are consuming propaganda and can't detect it as propaganda because it aligns with your values. However it subtly nudges your values in a direction over time. I agree that people's values and core beliefs are malleable, but in the same way a tree trunk is. It may seem like these people have changed a lot and you haven't, but I think it's more likely that you've changed too, and that they've changed less than you think. No one is immune to propaganda, which is why anything that can help disarm it interests me.
    - kubb 2y ago
      You touched on many points, but one thing to consider: people seek out information that confirms their worldview and actively protect themselves against anything that can harm their feelings for their idols. John Doe isn’t trained in logic and can adjust any of his premises if it means he can continue to admire his favorite celebrity. It’s a combination of flawed reasoning and premise flexibility. Not to mention, any fact can be endlessly challenged and questioned even if it’s agreed upon and largely incontestable.
naasking 2y ago
Humans are not fully rational, but they're more rational than many assume. For instance, many thought the illusory truth effect showed that people are biased towards believing things they hear many times over, which is great for propagandists, but it turns out this is only true when they are in a "high quality information" environment. This is quite rational! They should update towards believing repeated statements when the environment they're in has shown itself to be reliable. When the environment they're in has shown itself to be unreliable, the illusory truth effect basically disappears. [1] https://x.com/ROrchinik/status/1885820697160859951 https://x.com/ROrchinik/status/1885820697160859951
- kennysoona 2y ago
  How does that explain conservatives doubling down on whatever they hear even if it's obviously false? I guess because they wrongly consider some "low quality information" environments "high quality information" environments?
  - naasking 2y ago
    Not everything can be reduced to this one cognitive phenomenon. The behaviour you describe stems from: confirmation bias, and the backfire effect/identity-protective cognition. Also this isn't exclusive to conservatives: Science Denial Across the Political Divide: Liberals and Conservatives Are Similarly Motivated to Deny Attitude-Inconsistent Science, https://journals.sagepub.com/doi/abs/10.1177/1948550617731500?journalCode=sppa https://journals.sagepub.com/doi/abs/10.1177/194855061773150...
    - kennysoona 2y ago
      > Also this isn't exclusive to conservatives: You link aside, I think the obvious evidence is that that behavior is significantly more common in conservatives. Literally the most basic of facts get denied in bulk. I don't understand how you could make any argument that any other major political affiliation engages in the same behavior to a comparable extent.
aeturnum 2y ago
I think you're envisioning this in a pessimistic way. I totally agree that the end conclusion "this statement is fallacious" is pretty useless. But I assume that a working process would also yield the chain of judgements (A is right, B is right, C is wrong, etc). I think that would be VERY useful. People who become captured by propaganda and lies generally are not sold on 100% of the propaganda. There are certain elements they care more about and others they can ignore. A way to deprogram people through conversation is to just ask them to explain things about their views and ask them to reconcile them with reality. The reconciliation is painful for them and that pain keeps people "in" irrational beliefs - but it's also how people find their way out. Once they no longer associate themselves with the conspiracy, they can discard beliefs associated with it...provided they can think through them. I think being able to automatically decompose a fact check into the elements of what "is true" and "is false" in a statement would be HUGE. An essential tool in helping people escape from information swamps.
- RGamma 2y ago
  I vaguely remember a post I read on reddit [1] around the beginning of COVID by a nurse who dealt with an anti-vax patient. It went along the lines of "Big pharma wants to poison me", "Maybe you're being played and Chinese propaganda wants you to believe that to hurt the US". Apparently induced quite a lot of dissonance. Fighting fire with fire. [1] Impossible to find of course. And with all the LARPing going on on there, take this with two grains of salt. Given all the crazy shit going on in the US, I find it totally believable though.

FloorEgg 2y ago

I disagree with your first point. People are far more rational than you are making them out to be, it's just that they are rational within their own value system, not yours. Also today's propaganda is capable of adapting itself to each audience member's value system to make it more palatable, and then gradually nudge the audience towards the desired narrative/beliefs/values. The systems that distribute the propaganda are already analyzing people's values and using that information to manipulate people. I think that information asymmetry is part of the problem. I could be wrong, but I think flipping that dynamic around so the public can see the true values of the subjects of propaganda may help neutralize a lot of propaganda. As far as what impact this specific project will have, I have no idea. You may be right. I'm curious about its limitations and how it can be applied.

kubb 2y ago
I thought so too, but recently so many people dropped or adapted their core beliefs to be able to support and defend people in power that they really love that it made me change my mind. Now I think that value systems are malleable and are formed by whatever makes us feel good. And the logical consistency on top is very optional.
- FloorEgg 2y ago
  Or maybe these people you speak up assume that the people in power have values aligned with their own, and if there was an unbiased system that highlights value discrepancies using formal logic, they might not "love" those people as much anymore. What I assume you might be missing is that you are looking at the world through a different lens than these other people. Both you and they are consuming propaganda and can't detect it as propaganda because it aligns with your values. However it subtly nudges your values in a direction over time. I agree that people's values and core beliefs are malleable, but in the same way a tree trunk is. It may seem like these people have changed a lot and you haven't, but I think it's more likely that you've changed too, and that they've changed less than you think. No one is immune to propaganda, which is why anything that can help disarm it interests me.
  - kubb 2y ago
    You touched on many points, but one thing to consider: people seek out information that confirms their worldview and actively protect themselves against anything that can harm their feelings for their idols. John Doe isn’t trained in logic and can adjust any of his premises if it means he can continue to admire his favorite celebrity. It’s a combination of flawed reasoning and premise flexibility. Not to mention, any fact can be endlessly challenged and questioned even if it’s agreed upon and largely incontestable.

naasking 2y ago

Humans are not fully rational, but they're more rational than many assume. For instance, many thought the illusory truth effect showed that people are biased towards believing things they hear many times over, which is great for propagandists, but it turns out this is only true when they are in a "high quality information" environment. This is quite rational! They should update towards believing repeated statements when the environment they're in has shown itself to be reliable. When the environment they're in has shown itself to be unreliable, the illusory truth effect basically disappears. [1] https://x.com/ROrchinik/status/1885820697160859951 https://x.com/ROrchinik/status/1885820697160859951

kennysoona 2y ago
How does that explain conservatives doubling down on whatever they hear even if it's obviously false? I guess because they wrongly consider some "low quality information" environments "high quality information" environments?
- naasking 2y ago
  Not everything can be reduced to this one cognitive phenomenon. The behaviour you describe stems from: confirmation bias, and the backfire effect/identity-protective cognition. Also this isn't exclusive to conservatives: Science Denial Across the Political Divide: Liberals and Conservatives Are Similarly Motivated to Deny Attitude-Inconsistent Science, https://journals.sagepub.com/doi/abs/10.1177/1948550617731500?journalCode=sppa https://journals.sagepub.com/doi/abs/10.1177/194855061773150...
  - kennysoona 2y ago
    > Also this isn't exclusive to conservatives: You link aside, I think the obvious evidence is that that behavior is significantly more common in conservatives. Literally the most basic of facts get denied in bulk. I don't understand how you could make any argument that any other major political affiliation engages in the same behavior to a comparable extent.

aeturnum 2y ago

I think you're envisioning this in a pessimistic way. I totally agree that the end conclusion "this statement is fallacious" is pretty useless. But I assume that a working process would also yield the chain of judgements (A is right, B is right, C is wrong, etc). I think that would be VERY useful. People who become captured by propaganda and lies generally are not sold on 100% of the propaganda. There are certain elements they care more about and others they can ignore. A way to deprogram people through conversation is to just ask them to explain things about their views and ask them to reconcile them with reality. The reconciliation is painful for them and that pain keeps people "in" irrational beliefs - but it's also how people find their way out. Once they no longer associate themselves with the conspiracy, they can discard beliefs associated with it...provided they can think through them. I think being able to automatically decompose a fact check into the elements of what "is true" and "is false" in a statement would be HUGE. An essential tool in helping people escape from information swamps.

RGamma 2y ago
I vaguely remember a post I read on reddit [1] around the beginning of COVID by a nurse who dealt with an anti-vax patient. It went along the lines of "Big pharma wants to poison me", "Maybe you're being played and Chinese propaganda wants you to believe that to hurt the US". Apparently induced quite a lot of dissonance. Fighting fire with fire. [1] Impossible to find of course. And with all the LARPing going on on there, take this with two grains of salt. Given all the crazy shit going on in the US, I find it totally believable though.