15 ms·

Cognitive sophistication does not attenuate the bias blind spot (2012)

114 points by rzk 4y ago

largepeepee 4y ago
History is full of talented people who squandered their talents onto meaningless endeavours. Take the famed scientist Issac Newton for example, despite all his scientific knowledge and logical reasoning he wasted the second half of his life trying to turn other metals into gold, alchemy - based nothing more than biblical text. Big brain does not prevent ideological blindness.
- xyzzy123 4y ago
  Newton was a complex person and I think this is an overly reductive take on both his life and study of alchemy.
  - heavenlyblue 4y ago
    Is the next thing you are going to say that it's an overly reductive take on the alchemy as described in The Bible?
    - batch12 4y ago
      Where is this description? I am having a difficult time finding a reference to performing alchemy in the Bible.
      - h2odragon 4y ago
        That was one of the things that made alchemy so much fun; all the "instructions" were "encoded" and literally occult. Much of what they were doing was hunting for the keys to a known ciphertext (the Bible) that would give them the secrets of the universe, once they decoded it correctly. Of course we're far too bright and sophisticated to fall into such folly today. You won't fond modern peoples, for example, having passionate arguments about the meanings of text generated by mechanical / statistical processes.
  - deleted 4y ago
    [deleted]
- hprotagonist 4y ago
  a more apt vignette from newton’s life is that he absolutely lost his shirt on speculative investments in a bubble that collapsed and took one of the finest minds to the cleaners.
- marginalia_nu 4y ago
  > Take the famed scientist Issac Newton for example, despite all his scientific knowledge and logical reasoning he wasted the second half of his life trying to turn other metals into gold, alchemy - based nothing more than biblical text. Prior to the atomic model invented several hundred years later, given there was examples of metals turning into other metals (i.e. iron turning into steel, copper turning into brass)... I don't think there was much of a reason to think that the transmutation of metals into Gold wouldn't be possible. And it in fact sort of turns out to be possible in certain circumstances. We can make gold from lead (although it requires a particle accelerator).
  - zmgsabst 4y ago
    Alchemists being “wrong” is like Michelson-Morley “failing”: More branding than reality, since we accomplished their projects later — transmuting elements and detecting the aether. Ironically, this seems to be a cognitive bias — we never update our first impression nor put together a story across terminology changes. Carlo Rovelli gives an interesting talk on how we’re too quick to dismiss ancient scholars. https://www.youtube.com/watch?v=an5-KLvvMGM&t=1199s https://www.youtube.com/watch?v=an5-KLvvMGM&t=1199s
    - marginalia_nu 4y ago
      It's really fascinating to read historical accounts of science. Cicero's De Natura Deorum has an incredible account various antique world views and how they're motivated. Most of it is wrong in some sense in the light of modern science, but what's fascinating is how often they get things right. It's like they're using poorly chosen basis vectors, and muddling some concepts we now understand to be distinct: They seem to have associated energy and/or thermodynamic heat with the element of fire; the gaseous phase with air and buoyancy, the liquid phase with water, the solid phase with earth and gravity. It's very clear why the model stood for as long as it did despite some remarkably clever people looking into it. It actually fits the empirical evidence pretty well.
    - evv555 4y ago
      >More branding than reality, since we accomplished their projects later — transmuting elements and detecting the aether. Arguably this was accomplished using the work of alchemists. Ontologies tend to progress through stages of differentiation of components("stamp collecting") followed by integrative models. Mendeleev doesn't find periodic patterns in elements without alchemists first inventorying elemental features.
- stocknoob 4y ago
  After inventing modern physics and calculus I’d think I had the golden touch, too. Newton took a lot of intellectual paths, from tinkering with prisms to thinking about the paths of the heavens. Only looking back can we say “oh, you wasted your time investigating x and y, but z was right on. Why didn’t you just bat 1000?”
- mogacom 4y ago
  Historical context is important to take into account with Newton. He had no idea he was doing "science" as opposed to something else. He was trying to systematically investigate the natural world in an environment largely composed of ignorance and myth, which he was also part of. Wasn't alchemy simply chemistry before chemistry had been developed into a well-defined field of study? His obsession with the dimensions of Solomon's temple is a little hard to understand though!
- oh_sigh 4y ago
  It seems he was just overly credulous. Yes, he was very Christian, but I imagine if he got some ancient text from China on how to transmute lead to gold he would give it a shot
- tmn 4y ago
  I'll ignore the argument that alchemy was a precursor to modern science. An iron man version of 'lead to gold' is that it was never about literally turning lead to gold (which modern science can do). https://en.wikipedia.org/wiki/Alchemy https://en.wikipedia.org/wiki/Alchemy 'Esoteric interpretations of historical texts' From this lens, alchemy becomes a form of spiritual meditation. You can argue that such things are meaningless endeavors. But this is itself a subjective take. What are meaningful endeavors really? If I want to take a nihilistic frame, I'll strengthen your comment and say it's impossible to avoid meaningless endeavors. Inevitably one must pick their bias and arrive at their narrative.
- asdff 4y ago
  Newton was right that you could turn lead into gold, but it wasn't his fault that nuclear science needed to advance a few hundred years before someone ran that actual experiment.
boomboomsubban 4y ago
>Additional analyses indicated that being free of the bias blind spot does not help a person avoid the actual classic cognitive biases. This is the worst part. Being fully aware that people are biased and that you are not a special exception to that isn't helpful. You're still just as likely to let bias influence your actions, and you'll probably try to justify yourself as having good reasons for your biased actions.
- yamrzou 4y ago
  That's why we need feedback from others. “A system cannot understand itself without help from outside the system, because prior experiences will bias objectivity, preventing critical analysis of the organization.” — Edward Deming
  - luckylion 4y ago
    How often can you get feedback from someone until they've become part of your system and are no longer an outsider?
    - Byamarro 4y ago
      This is such a valid point. I've experienced it myself. When you're on a team, after some time team members start developing shared biases. It's enough that some previous discussion was railroaded certain way to make it harder for the team members to think outside of the box shaped by previous conversations.
  - aatd86 4y ago
    Until you realize that part of you is a black box to others and only you have access to some of your internal state. So, yes and no. That's why self awareness, perhaps via proper meditation, although I thought it was some hippy shit at first, is actually valuable.
- yodsanklai 4y ago
  > You're still just as likely to let bias influence your actions, and you'll probably try to justify yourself as having good reasons for your biased actions. I really don't think everybody is equally biased. Some people at least try to use reason and science to guide their beliefs, why other will systematically deny all evidences. Also people who are aware of biases are more likely to recognize their errors when called out with logical arguments.
michaelt 4y ago
Available from the coauthor's website at http://www.keithstanovich.com/Site/Research_on_Reasoning_files/West_Stanovich_JPSP2012.pdf http://www.keithstanovich.com/Site/Research_on_Reasoning_fil...
qikInNdOutReply 4y ago
There is a whole interleafing flower petal of mental bug categorys: Detectable | Undetectable Fixable | Unfixable Workaround | No-Workaround Cost of workaround (Big O) Individual | Society Non-Meta | Meta (Bug is created by several other bugs recombining into a functional or dysfunctional cluster) Some bugs you cant fix, cause society depends on it while also suffering with it. Humanity sized Behaviour DBs and Datamining is the real Psychology!
tootie 4y ago
My layman observation was always that being intelligent and being rational are two completely different things. It's how you get people who have high technical achievement but still hold on to odd beliefs or get sucked into conspiracies.
1letterunixname 4y ago
As the bumper sticker says: "No shit." Intelligence in one domain does not necessarily translate to all areas. There are uncountable millions of gifted people who are low functioning in other domains. Contrary to this are the individual fallacies of overconfidence and the collective fallacy of the genius halo effect. There is also a tendency to failure aversion by avoid domains outside of performant ones. The safest assumption is: 1. Your areas outside of performant ones are lacking until proven otherwise with feedback. 2. Don't worry about not being an omnipotent polyglot, polymath. Seek excellence, just don't go crazy or drive everyone else nuts. Theory of mind and frame taking by considering putting yourself in others' proverbial shoes isn't necessarily something people are born with, it has to be taught and learned. Some people do have harder times because they have to intellectualize it if they are emotionally disconnected from others and/or themselves, or have a neurotype that doesn't lend itself to smooth observation of social and situational state of others.
- peatmoss 4y ago
  This is one of those things where I’ve lived the XKCD comic (https://xkcd.com/793/ https://xkcd.com/793/). I once had a job where I was part of the supporting technical crew for some very smart people with some truly impressive accomplishments. A colleague and I sometimes groused about the challenges of working with “universal domain specialists.”
  - dkarl 4y ago
    Tests and version control... why do you bother with all that ceremony? It seems like a lot of work that could go towards making the code better. To play devil's advocate, though, I do think that something is lost if everyone stops themselves from asking questions on the grounds that everyone has good reasons for everything they do and therefore there's no point having thoughts about someone else's job. Even if you assume the first part, the second doesn't follow. Understanding other people's jobs helps you work better with them, and if you don't understand the purpose of something they're doing, that probably indicates an opportunity to understand them better. Second, looking at it from the other side, we all benefit from answering beginner questions. What's the difference between the innocence of a child and the innocence of somebody who has studied physics their whole life and never had to support a piece of software? When children, or physicists, ask questions that are hard for us to answer, we can learn something in the attempt. The only difference between being posed "dumb" questions by a child and an adult from another field is if they think you're an idiot. (Honestly, that isn't even a difference. We just don't mind if children walk around thinking adults are all idiots most of the time. We expect differently from them after they grow up.) Reading between the lines, I think some people regard the momentary joy of thinking you see something that someone else doesn't as a bit of corruption in the soul, a moral root cause. I don't think it's a bad thing, though. Imagine playing chess against someone much better than you and getting excited because you think you can take a piece and get a winning advantage. The odds are very high that you are wrong, or that you're going to take the piece and then lose anyway, but what are you going to do? You're going to take the piece, pose them the question! Or else there's no fun in playing the game. Some people would argue that the excitement that arises in your mind at that point is wrong, not just in the sense of not being supported by reason, but a morally wrong response, arrogant, disrespectful of the better player, a toxic feeling that we should strive to eliminate and certainly strive not to show to other people. But it isn't necessarily, and we don't have to take it that way. Where it can change from innocent to annoying and even toxic, depending on how people handle it, is when the expert can't provide an answer that satisfies the innocent. At that point the innocent (or the expert) might get angry or condescending, or might simply badger the other person in a time-wasting way, insisting that the matter can't be dropped until agreement is reached. Or one or both of them might walk away thinking the other person is an idiot. But that doesn't have to happen. They can let it go gracefully. The innocent can walk away thinking, "I don't understand their reason for doing that," the expert can walk away thinking, "I don't know how to explain it to someone with their background," and they both have food for further thought.
maCDzP 4y ago
How does members of the HN crowd try to combat this? Or is it just another exercise in futility where the actual exercise is just part of the thing that make me biased in the first place? I’ll go first. I see a therapist every other week. I hope that gives me an outsiders point of view. But I think I have identified a way to know where my biases are could be strong. When I get a really uneasy feeling when I start to think about making changes in a certain area. And I don’t know if it’s a rational fear or just a sunk cost fallacy. What I find really scary is that I have hard time understanding if I am just fooling myself or being rational. Because it’s always my own mind coming up with explanations, so the actual process looks the same to me. I read a book called “Quit” recently and it gave me a lot to think about regarding biases and quitting different endeavors. Can recommend.
- SamoyedFurFluff 4y ago
  I would say isn’t it useful to simply acknowledge you can be deeply flawed all the time and your experiences of reality will always be slanted with your subjective opinions on how society works, that everyone else is also doing the same thing, and continue onwards anyways? There’s no way to escape our own imperfections. We have some tools to help us mitigate them, but they will always be there.
  - brookst 4y ago
    That seems fairly defeatist. I’ve gotten better at some things I used to be terrible at, and it took work to do so. And I’m happier for it, while still being plenty imperfect.
    - SamoyedFurFluff 4y ago
      I never said to stop working at it. I’m merely pointing out that the knowledge that one is a personally unsolveably flawed human being doesn’t have to cause uneasiness.
      - brookst 4y ago
blfr 4y ago
Are these biases wrong though? In a hypothetical perfectly spherical chickens in vacuum world, yes, but in reality you probably want some outcome bias for example because you don't control all the variables. So the analysis of perception of that bias and bias blind spot strike me as interesting but rather useless.
- naasking 4y ago
  They're wrong if you're interested in truth, just maybe not wrong if you're interested in reproductive fitness. We're not so concerned about the latter these days so much as the former.
- lumb63 4y ago
  Your comment gave me an interesting way to think about biases: Bayesian statistics. It is no wonder we see people end up at vastly different conclusions when their starting point (bias) is so different. Coupled with this, bias is probably correlated with experiences to some extent, leading two different individuals to entirely different conclusions. This is a good argument for devolution, IMO. Today’s politics is too large-scale and winner-take-all to accurately represent all its constituents. For instance, a topic like immigration in the US has no solution that will appease everyone, because there are so many different viewpoints that conflict with each other. We should let smaller populations with a higher level of concurrence govern themselves accordingly.
  - asdff 4y ago
    On the other hand, maybe the only population size sufficient to have a high level of concurrence in thought is 1 individual. Maybe there are weird effects going on with individual preferences as population gets small. Consider you and a spouse deciding on dinner, you want pasta they want tacos. Complete disagreement. Imagine you now have a kid, they want tacos too. Maybe given the population size, you soften your convictions on the pasta, and opt for the tacos since the majority is leaning that way. A phenomenon like this, whether it is real or not, would screw up the small pop is more agreeable model, for example.
xyzelement 4y ago
I observe something like this in the way people react to what's going on in the news. Example: compared to most people, I am "better" positioned to understand what's going on in Ukraine - my dad and both inlaws are from there and speak the language. I grew up in former USSR and speak fluent Russian so I can also watch and read things closer to the source. And yet, I don't claim to understand anything that's going on in Ukraine because ~none~ of the information I see can be considered unbiased. So I can chose to believe one set but not the other set of data or I can say "I am not equipped to understand this." Meanwhile, tons of people who are objectively less equipped to suss out the situation, don't feel this hesitation to have an opinion. Intellectually, if you ask them 'does propaganda exist, and is it amped up in a war situation' they will say 'of course, every time' and then they'll go and believe what they read (be in in a Western or Russian source, doesn't matter.) This is not a complaint, it's actually something quite empowering to understand.
- ag315 4y ago
  This is a really excellent point, both in general and vis a vis the conflict in Ukraine. Something I see happening often is that some people are quick to confuse expertise-as-credential with expertise-as-substance, or to assume that anyone who has above average knowledge of some relevant subject area that relates to some aspect of the war is immune to emotionally biased interpretations.
- kulahan 4y ago
  This was recently on display with the Ghost of Kyiv killing over 850,000 enemy platoons with a single bullet. I'm sure it was born out of some interesting story - maybe a rookie had a good dogfight or something, but BOY did that story grow and grow. My only question is: how much propaganda came from states, vs. people taking an idea and running with it because it's a narrative they want to hear?
  - 8f2ab37a-ed6c 4y ago
    Or the rumors of mobile cremation chambers to incinerate corpses of crimes against humanity victims which circulated in the first few weeks, including copious photos, which all turned out to be some random heavy equipment.
deleted 4y ago
[deleted]
apienx 4y ago
It'd be interesting to see how the Skeptics and other bias-aware groups perform. It's hard to imagine the folks at the Center For Applied Rationally doing just as bad as the average Joe. It's hard to believe that you can't train yourself into hypervigilance and extraordinary awareness. Practicing with intent is a very different animal from "cognitive sophistication" (however the paper defines it).
- 8f2ab37a-ed6c 4y ago
  The science seems to show that it's very hard (possibly almost impossible) to de-bias yourself, which is why humans have to rely rely on other humans with outside perspective as a counter-balancing force. This assumes that conflict of opinions is welcome and accepted by the group, which is unfortunately not always the case, leading to echo chambers and groups confirming and reinforcing existing beliefs.
- jgeada 4y ago
  You cannot be an expert in everything, it is impossible; the domain of human knowledge and experience has grown too vast for anyone to be conversant with anything more than a minor fraction. So, sooner or later, either you distrust everything and you become a conspiracy theorist or you rely on experts. Who you choose to trust as experts gives you a bias. My personal observation (ie read this as anecdote, not data) is that skeptics frequently choose to select as their expert the lone dissenter in the field. My interpretation is not that the skeptics I know are truly skeptics, they're contrarians who don't trust institutional knowledge or consensus.
  - asdff 4y ago
    Chomsky writes about the expert effect in Manufacturing Consent. Basically, among credentialed experts on any given topic, you can find opinions going every which way. So bringing in an expert to bolster your point is meaningless, because they are brought in because of their preexisting opinions which as you note sometimes run against the rest of the field. Another technique is to bring in someone with contrary opinions and one with favorable opinions, and basically not give them much shot to speak or force them on the defensive for everything they say with both your expert and the moderator. Then you can say you soundly defeated that critical opinion of the topic.
at_a_remove 4y ago
My methodology, such as it is, goes like: 1) Receive information. 2) Assume that I am being lied to, manipulated, shaded, data elided, whatever, sometimes by accident, sometimes by intent, but at least several levels removed from whomever delivers the information. 3) Seek out information that says the exact opposite. 4) Repeat Step 2. 5) Start looking for little tricks: what has been left out? If we are judging person X for act Y, are we not judging other parties for Y? Is the timing convenient for a distraction? 6) Resign myself to a state of uncertainty in a world of lies.
2devnull 4y ago
A lot of great comments here about the difficulty recognizing one’s own biases. A related thing I like to think about is the jungian concept of the shadow; if we can see it, it’s not the shadow. I do think we can look for evidence that we are at least limiting the impact of our biases. For example, if you don’t change political affiliations at least once or twice in your lifetime, that’s probably a sign you’re controlled by your biases and not really as free or rational as is optimal. What’s the old saying, if you aren’t a liberal in your youth you have no heart; if you aren’t conservative as you get older you have no brain. But I think moving in the opposite direction is also a good indicator of free thinking. A foolish consistency is the hob goblin of little minds. Dynamic thinking is the only real thinking. We should grow and evolve overtime.
golemiprague 4y ago
[dead]
nate_770 4y ago
http://www.keithstanovich.com/Site/Research_on_Reasoning_files/West_Stanovich_JPSP2012.pdf http://www.keithstanovich.com/Site/Research_on_Reasoning_fil... Here’s a link to the full study.

114 points by rzk 4y ago

History is full of talented people who squandered their talents onto meaningless endeavours. Take the famed scientist Issac Newton for example, despite all his scientific knowledge and logical reasoning he wasted the second half of his life trying to turn other metals into gold, alchemy - based nothing more than biblical text. Big brain does not prevent ideological blindness.

xyzzy123 4y ago
Newton was a complex person and I think this is an overly reductive take on both his life and study of alchemy.
- heavenlyblue 4y ago
  Is the next thing you are going to say that it's an overly reductive take on the alchemy as described in The Bible?
  - batch12 4y ago
    Where is this description? I am having a difficult time finding a reference to performing alchemy in the Bible.
    - h2odragon 4y ago
      That was one of the things that made alchemy so much fun; all the "instructions" were "encoded" and literally occult. Much of what they were doing was hunting for the keys to a known ciphertext (the Bible) that would give them the secrets of the universe, once they decoded it correctly. Of course we're far too bright and sophisticated to fall into such folly today. You won't fond modern peoples, for example, having passionate arguments about the meanings of text generated by mechanical / statistical processes.
- deleted 4y ago
  [deleted]
hprotagonist 4y ago
a more apt vignette from newton’s life is that he absolutely lost his shirt on speculative investments in a bubble that collapsed and took one of the finest minds to the cleaners.
marginalia_nu 4y ago
> Take the famed scientist Issac Newton for example, despite all his scientific knowledge and logical reasoning he wasted the second half of his life trying to turn other metals into gold, alchemy - based nothing more than biblical text. Prior to the atomic model invented several hundred years later, given there was examples of metals turning into other metals (i.e. iron turning into steel, copper turning into brass)... I don't think there was much of a reason to think that the transmutation of metals into Gold wouldn't be possible. And it in fact sort of turns out to be possible in certain circumstances. We can make gold from lead (although it requires a particle accelerator).
- zmgsabst 4y ago
  Alchemists being “wrong” is like Michelson-Morley “failing”: More branding than reality, since we accomplished their projects later — transmuting elements and detecting the aether. Ironically, this seems to be a cognitive bias — we never update our first impression nor put together a story across terminology changes. Carlo Rovelli gives an interesting talk on how we’re too quick to dismiss ancient scholars. https://www.youtube.com/watch?v=an5-KLvvMGM&t=1199s https://www.youtube.com/watch?v=an5-KLvvMGM&t=1199s
  - marginalia_nu 4y ago
    It's really fascinating to read historical accounts of science. Cicero's De Natura Deorum has an incredible account various antique world views and how they're motivated. Most of it is wrong in some sense in the light of modern science, but what's fascinating is how often they get things right. It's like they're using poorly chosen basis vectors, and muddling some concepts we now understand to be distinct: They seem to have associated energy and/or thermodynamic heat with the element of fire; the gaseous phase with air and buoyancy, the liquid phase with water, the solid phase with earth and gravity. It's very clear why the model stood for as long as it did despite some remarkably clever people looking into it. It actually fits the empirical evidence pretty well.
  - evv555 4y ago
    >More branding than reality, since we accomplished their projects later — transmuting elements and detecting the aether. Arguably this was accomplished using the work of alchemists. Ontologies tend to progress through stages of differentiation of components("stamp collecting") followed by integrative models. Mendeleev doesn't find periodic patterns in elements without alchemists first inventorying elemental features.
stocknoob 4y ago
After inventing modern physics and calculus I’d think I had the golden touch, too. Newton took a lot of intellectual paths, from tinkering with prisms to thinking about the paths of the heavens. Only looking back can we say “oh, you wasted your time investigating x and y, but z was right on. Why didn’t you just bat 1000?”
mogacom 4y ago
Historical context is important to take into account with Newton. He had no idea he was doing "science" as opposed to something else. He was trying to systematically investigate the natural world in an environment largely composed of ignorance and myth, which he was also part of. Wasn't alchemy simply chemistry before chemistry had been developed into a well-defined field of study? His obsession with the dimensions of Solomon's temple is a little hard to understand though!
oh_sigh 4y ago
It seems he was just overly credulous. Yes, he was very Christian, but I imagine if he got some ancient text from China on how to transmute lead to gold he would give it a shot
tmn 4y ago
I'll ignore the argument that alchemy was a precursor to modern science. An iron man version of 'lead to gold' is that it was never about literally turning lead to gold (which modern science can do). https://en.wikipedia.org/wiki/Alchemy https://en.wikipedia.org/wiki/Alchemy 'Esoteric interpretations of historical texts' From this lens, alchemy becomes a form of spiritual meditation. You can argue that such things are meaningless endeavors. But this is itself a subjective take. What are meaningful endeavors really? If I want to take a nihilistic frame, I'll strengthen your comment and say it's impossible to avoid meaningless endeavors. Inevitably one must pick their bias and arrive at their narrative.
asdff 4y ago
Newton was right that you could turn lead into gold, but it wasn't his fault that nuclear science needed to advance a few hundred years before someone ran that actual experiment.

boomboomsubban 4y ago

>Additional analyses indicated that being free of the bias blind spot does not help a person avoid the actual classic cognitive biases. This is the worst part. Being fully aware that people are biased and that you are not a special exception to that isn't helpful. You're still just as likely to let bias influence your actions, and you'll probably try to justify yourself as having good reasons for your biased actions.

yamrzou 4y ago
That's why we need feedback from others. “A system cannot understand itself without help from outside the system, because prior experiences will bias objectivity, preventing critical analysis of the organization.” — Edward Deming
- luckylion 4y ago
  How often can you get feedback from someone until they've become part of your system and are no longer an outsider?
  - Byamarro 4y ago
    This is such a valid point. I've experienced it myself. When you're on a team, after some time team members start developing shared biases. It's enough that some previous discussion was railroaded certain way to make it harder for the team members to think outside of the box shaped by previous conversations.
- aatd86 4y ago
  Until you realize that part of you is a black box to others and only you have access to some of your internal state. So, yes and no. That's why self awareness, perhaps via proper meditation, although I thought it was some hippy shit at first, is actually valuable.
yodsanklai 4y ago
> You're still just as likely to let bias influence your actions, and you'll probably try to justify yourself as having good reasons for your biased actions. I really don't think everybody is equally biased. Some people at least try to use reason and science to guide their beliefs, why other will systematically deny all evidences. Also people who are aware of biases are more likely to recognize their errors when called out with logical arguments.

michaelt 4y ago

Available from the coauthor's website at http://www.keithstanovich.com/Site/Research_on_Reasoning_files/West_Stanovich_JPSP2012.pdf http://www.keithstanovich.com/Site/Research_on_Reasoning_fil...

qikInNdOutReply 4y ago

There is a whole interleafing flower petal of mental bug categorys: Detectable | Undetectable Fixable | Unfixable Workaround | No-Workaround Cost of workaround (Big O) Individual | Society Non-Meta | Meta (Bug is created by several other bugs recombining into a functional or dysfunctional cluster) Some bugs you cant fix, cause society depends on it while also suffering with it. Humanity sized Behaviour DBs and Datamining is the real Psychology!

tootie 4y ago

My layman observation was always that being intelligent and being rational are two completely different things. It's how you get people who have high technical achievement but still hold on to odd beliefs or get sucked into conspiracies.

1letterunixname 4y ago

As the bumper sticker says: "No shit." Intelligence in one domain does not necessarily translate to all areas. There are uncountable millions of gifted people who are low functioning in other domains. Contrary to this are the individual fallacies of overconfidence and the collective fallacy of the genius halo effect. There is also a tendency to failure aversion by avoid domains outside of performant ones. The safest assumption is: 1. Your areas outside of performant ones are lacking until proven otherwise with feedback. 2. Don't worry about not being an omnipotent polyglot, polymath. Seek excellence, just don't go crazy or drive everyone else nuts. Theory of mind and frame taking by considering putting yourself in others' proverbial shoes isn't necessarily something people are born with, it has to be taught and learned. Some people do have harder times because they have to intellectualize it if they are emotionally disconnected from others and/or themselves, or have a neurotype that doesn't lend itself to smooth observation of social and situational state of others.

peatmoss 4y ago
This is one of those things where I’ve lived the XKCD comic (https://xkcd.com/793/ https://xkcd.com/793/). I once had a job where I was part of the supporting technical crew for some very smart people with some truly impressive accomplishments. A colleague and I sometimes groused about the challenges of working with “universal domain specialists.”
- dkarl 4y ago
  Tests and version control... why do you bother with all that ceremony? It seems like a lot of work that could go towards making the code better. To play devil's advocate, though, I do think that something is lost if everyone stops themselves from asking questions on the grounds that everyone has good reasons for everything they do and therefore there's no point having thoughts about someone else's job. Even if you assume the first part, the second doesn't follow. Understanding other people's jobs helps you work better with them, and if you don't understand the purpose of something they're doing, that probably indicates an opportunity to understand them better. Second, looking at it from the other side, we all benefit from answering beginner questions. What's the difference between the innocence of a child and the innocence of somebody who has studied physics their whole life and never had to support a piece of software? When children, or physicists, ask questions that are hard for us to answer, we can learn something in the attempt. The only difference between being posed "dumb" questions by a child and an adult from another field is if they think you're an idiot. (Honestly, that isn't even a difference. We just don't mind if children walk around thinking adults are all idiots most of the time. We expect differently from them after they grow up.) Reading between the lines, I think some people regard the momentary joy of thinking you see something that someone else doesn't as a bit of corruption in the soul, a moral root cause. I don't think it's a bad thing, though. Imagine playing chess against someone much better than you and getting excited because you think you can take a piece and get a winning advantage. The odds are very high that you are wrong, or that you're going to take the piece and then lose anyway, but what are you going to do? You're going to take the piece, pose them the question! Or else there's no fun in playing the game. Some people would argue that the excitement that arises in your mind at that point is wrong, not just in the sense of not being supported by reason, but a morally wrong response, arrogant, disrespectful of the better player, a toxic feeling that we should strive to eliminate and certainly strive not to show to other people. But it isn't necessarily, and we don't have to take it that way. Where it can change from innocent to annoying and even toxic, depending on how people handle it, is when the expert can't provide an answer that satisfies the innocent. At that point the innocent (or the expert) might get angry or condescending, or might simply badger the other person in a time-wasting way, insisting that the matter can't be dropped until agreement is reached. Or one or both of them might walk away thinking the other person is an idiot. But that doesn't have to happen. They can let it go gracefully. The innocent can walk away thinking, "I don't understand their reason for doing that," the expert can walk away thinking, "I don't know how to explain it to someone with their background," and they both have food for further thought.

maCDzP 4y ago

How does members of the HN crowd try to combat this? Or is it just another exercise in futility where the actual exercise is just part of the thing that make me biased in the first place? I’ll go first. I see a therapist every other week. I hope that gives me an outsiders point of view. But I think I have identified a way to know where my biases are could be strong. When I get a really uneasy feeling when I start to think about making changes in a certain area. And I don’t know if it’s a rational fear or just a sunk cost fallacy. What I find really scary is that I have hard time understanding if I am just fooling myself or being rational. Because it’s always my own mind coming up with explanations, so the actual process looks the same to me. I read a book called “Quit” recently and it gave me a lot to think about regarding biases and quitting different endeavors. Can recommend.

SamoyedFurFluff 4y ago
I would say isn’t it useful to simply acknowledge you can be deeply flawed all the time and your experiences of reality will always be slanted with your subjective opinions on how society works, that everyone else is also doing the same thing, and continue onwards anyways? There’s no way to escape our own imperfections. We have some tools to help us mitigate them, but they will always be there.
- brookst 4y ago
  That seems fairly defeatist. I’ve gotten better at some things I used to be terrible at, and it took work to do so. And I’m happier for it, while still being plenty imperfect.
  - SamoyedFurFluff 4y ago
    I never said to stop working at it. I’m merely pointing out that the knowledge that one is a personally unsolveably flawed human being doesn’t have to cause uneasiness.
    - brookst 4y ago

blfr 4y ago

Are these biases wrong though? In a hypothetical perfectly spherical chickens in vacuum world, yes, but in reality you probably want some outcome bias for example because you don't control all the variables. So the analysis of perception of that bias and bias blind spot strike me as interesting but rather useless.

naasking 4y ago
They're wrong if you're interested in truth, just maybe not wrong if you're interested in reproductive fitness. We're not so concerned about the latter these days so much as the former.
lumb63 4y ago
Your comment gave me an interesting way to think about biases: Bayesian statistics. It is no wonder we see people end up at vastly different conclusions when their starting point (bias) is so different. Coupled with this, bias is probably correlated with experiences to some extent, leading two different individuals to entirely different conclusions. This is a good argument for devolution, IMO. Today’s politics is too large-scale and winner-take-all to accurately represent all its constituents. For instance, a topic like immigration in the US has no solution that will appease everyone, because there are so many different viewpoints that conflict with each other. We should let smaller populations with a higher level of concurrence govern themselves accordingly.
- asdff 4y ago
  On the other hand, maybe the only population size sufficient to have a high level of concurrence in thought is 1 individual. Maybe there are weird effects going on with individual preferences as population gets small. Consider you and a spouse deciding on dinner, you want pasta they want tacos. Complete disagreement. Imagine you now have a kid, they want tacos too. Maybe given the population size, you soften your convictions on the pasta, and opt for the tacos since the majority is leaning that way. A phenomenon like this, whether it is real or not, would screw up the small pop is more agreeable model, for example.

xyzelement 4y ago

I observe something like this in the way people react to what's going on in the news. Example: compared to most people, I am "better" positioned to understand what's going on in Ukraine - my dad and both inlaws are from there and speak the language. I grew up in former USSR and speak fluent Russian so I can also watch and read things closer to the source. And yet, I don't claim to understand anything that's going on in Ukraine because ~none~ of the information I see can be considered unbiased. So I can chose to believe one set but not the other set of data or I can say "I am not equipped to understand this." Meanwhile, tons of people who are objectively less equipped to suss out the situation, don't feel this hesitation to have an opinion. Intellectually, if you ask them 'does propaganda exist, and is it amped up in a war situation' they will say 'of course, every time' and then they'll go and believe what they read (be in in a Western or Russian source, doesn't matter.) This is not a complaint, it's actually something quite empowering to understand.

ag315 4y ago
This is a really excellent point, both in general and vis a vis the conflict in Ukraine. Something I see happening often is that some people are quick to confuse expertise-as-credential with expertise-as-substance, or to assume that anyone who has above average knowledge of some relevant subject area that relates to some aspect of the war is immune to emotionally biased interpretations.
kulahan 4y ago
This was recently on display with the Ghost of Kyiv killing over 850,000 enemy platoons with a single bullet. I'm sure it was born out of some interesting story - maybe a rookie had a good dogfight or something, but BOY did that story grow and grow. My only question is: how much propaganda came from states, vs. people taking an idea and running with it because it's a narrative they want to hear?
- 8f2ab37a-ed6c 4y ago
  Or the rumors of mobile cremation chambers to incinerate corpses of crimes against humanity victims which circulated in the first few weeks, including copious photos, which all turned out to be some random heavy equipment.

deleted 4y ago

[deleted]

apienx 4y ago

It'd be interesting to see how the Skeptics and other bias-aware groups perform. It's hard to imagine the folks at the Center For Applied Rationally doing just as bad as the average Joe. It's hard to believe that you can't train yourself into hypervigilance and extraordinary awareness. Practicing with intent is a very different animal from "cognitive sophistication" (however the paper defines it).

8f2ab37a-ed6c 4y ago
The science seems to show that it's very hard (possibly almost impossible) to de-bias yourself, which is why humans have to rely rely on other humans with outside perspective as a counter-balancing force. This assumes that conflict of opinions is welcome and accepted by the group, which is unfortunately not always the case, leading to echo chambers and groups confirming and reinforcing existing beliefs.
jgeada 4y ago
You cannot be an expert in everything, it is impossible; the domain of human knowledge and experience has grown too vast for anyone to be conversant with anything more than a minor fraction. So, sooner or later, either you distrust everything and you become a conspiracy theorist or you rely on experts. Who you choose to trust as experts gives you a bias. My personal observation (ie read this as anecdote, not data) is that skeptics frequently choose to select as their expert the lone dissenter in the field. My interpretation is not that the skeptics I know are truly skeptics, they're contrarians who don't trust institutional knowledge or consensus.
- asdff 4y ago
  Chomsky writes about the expert effect in Manufacturing Consent. Basically, among credentialed experts on any given topic, you can find opinions going every which way. So bringing in an expert to bolster your point is meaningless, because they are brought in because of their preexisting opinions which as you note sometimes run against the rest of the field. Another technique is to bring in someone with contrary opinions and one with favorable opinions, and basically not give them much shot to speak or force them on the defensive for everything they say with both your expert and the moderator. Then you can say you soundly defeated that critical opinion of the topic.

at_a_remove 4y ago

My methodology, such as it is, goes like: 1) Receive information. 2) Assume that I am being lied to, manipulated, shaded, data elided, whatever, sometimes by accident, sometimes by intent, but at least several levels removed from whomever delivers the information. 3) Seek out information that says the exact opposite. 4) Repeat Step 2. 5) Start looking for little tricks: what has been left out? If we are judging person X for act Y, are we not judging other parties for Y? Is the timing convenient for a distraction? 6) Resign myself to a state of uncertainty in a world of lies.

2devnull 4y ago

A lot of great comments here about the difficulty recognizing one’s own biases. A related thing I like to think about is the jungian concept of the shadow; if we can see it, it’s not the shadow. I do think we can look for evidence that we are at least limiting the impact of our biases. For example, if you don’t change political affiliations at least once or twice in your lifetime, that’s probably a sign you’re controlled by your biases and not really as free or rational as is optimal. What’s the old saying, if you aren’t a liberal in your youth you have no heart; if you aren’t conservative as you get older you have no brain. But I think moving in the opposite direction is also a good indicator of free thinking. A foolish consistency is the hob goblin of little minds. Dynamic thinking is the only real thinking. We should grow and evolve overtime.

golemiprague 4y ago

[dead]

nate_770 4y ago

http://www.keithstanovich.com/Site/Research_on_Reasoning_files/West_Stanovich_JPSP2012.pdf http://www.keithstanovich.com/Site/Research_on_Reasoning_fil... Here’s a link to the full study.