10 ms·

When rational thinking is correlated with intelligence the correlation is modest

120 points by lkrubner 12y ago

lkrubner 12y ago
I consider myself smart and rational, so I was surprised that I stumbled on many of the test questions. I got this one wrong, though you would think, having been programming computers for 15 years, I surely should have gotten this one right: Jack is looking at Anne, but Anne is looking at George. Jack is married, but George is not. Is a married person looking at an unmarried person? A) Yes B) No C) Cannot be determined Oddly enough, I feel that if this had been presented to me as a bit of Java or Ruby code, then I would have gotten this one correct. It's the informality of English, and the assumptions I make when reading English, that tripped me up. Computer code engages the rational side of my brain in a way that English does not. And to follow up on a point made long ago by Jeff Atwood, I think this (that IQ and rationality are different) also explains the strange inability of some smart people to learn how to program computers. I'm thinking of the post here: http://blog.codinghorror.com/separating-programming-sheep-from-non-programming-goats/ http://blog.codinghorror.com/separating-programming-sheep-fr... Jeff Atwood discusses this quote: "All teachers of programming find that their results display a 'double hump'. It is as if there are two populations: those who can [program], and those who cannot [program], each with its own independent bell curve. Almost all research into programming teaching and learning have concentrated on teaching: change the language, change the application area, use an IDE and work on motivation. None of it works, and the double hump persists." And also this quote: "Despite the enormous changes which have taken place since electronic computing was invented in the 1950s, some things remain stubbornly the same. In particular, most people can't learn to program: between 30% and 60% of every university computer science department's intake fail the first programming course. Experienced teachers are weary but never oblivious of this fact; brighteyed beginners who believe that the old ones must have been doing it wrong learn the truth from bitter experience; and so it has been for almost two generations, ever since the subject began in the 1960s."
- Rainymood 12y ago
  SPOILER ALERT (ANSWER OF RIDDLE BELOW): I don't really see where you got stuck. Could you mind explaining me why/how/where you got stuck? I couldn't find a satisfying 'aha!' moment so I just kind of brute forced all the possibilities ..
  - mbell 12y ago
    I got it wrong, not due to lack of logic but due to lack of spending the time to untangle the strange wording. I read it as "Jack is looking at Anne, but Anne is looking at _Jack_." and assumed it was some sort of trick question. If they were really interested in testing logic they would deliver the problem in a clear manor rather than masking it strange wording.
  - lkrubner 12y ago
    SPOILER ALERT (ANSWER OF RIDDLE BELOW): Like most people, my first thought was c. And yet, as I said, I suspect that if this had been presented as a bit of Java or Ruby, I would have been unlikely to make that mistake. I read English with a laziness that is different from how I read computer code.
    - deleted 12y ago
      [deleted]
  - gjm11 12y ago
    The failure mode isn't getting stuck, it's failing to notice that there's actual work to do. The way you get this one wrong is a thought process like this: "Well, obviously we only know about the first and third links in the chain, and we aren't told anything about the second one. So of course we can't tell whether there's a married person looking at an unmarried one because we have no information about any two directly 'connected' people." (But less explicit -- if you think about it in that much detail you're likely to think of brute-forcing the possibilities.) Or like this: "Well, let's see: A looks at B looks at C, A is married, C not. Does A->B have to be married->unmarried? No; B could be married. Does B->C have to be married->unmarried? No; B could be unmarried. So we're done." Once it occurs to you that you could just kind of brute force all the possibilities, you're going to get it right. The problem comes when you short-circuit the process. (And the reason why doing so isn't just stupid is that often this sort of "pruning" of mental effort is key to working things out efficiently. It just happens that in this case the prunability is illusory.)
pessimizer 12y ago
Only if you define intelligence as IQ is this true. It's more accurate to say that scoring high on IQ tests doesn't correlate very strongly with reasoning skills. Or, more interestingly, that scores on IQ tests correlate far more with factors like parental incomes or pleasure reading during childhood than they do with reasoning skills.
UhUhUhUh 12y ago
Most IQ tests have various scales and subscales, verbal, non-verbal, speed, memory etc... Aside from the few subtests that measure fund of knowledge, speed of reasoning, pattern extraction/recognition and ability to abstract problems define (loosely) what is being measured as "intelligence." The "G" Factor. Note that none of these categories strictly implies rational thinking. You can even make a point that scoring high on any of these categories will make you less likely to make rational decisions.
- sysk 12y ago
  <rant>It seems there's a lot of parallels to be made between IQ tests and programming language micro-benchmarks. Also, they seem to be equally useful: that is, not very except at the extremes. The corollary is that if you didn't pick your programming language based on a micro-benchmark, you probably shouldn't give too much weight to IQ when hiring.
deleted 12y ago
[deleted]
tokenadult 12y ago
The author of the article kindly submitted to open this thread has thought deeply about the title issue of the article and has written a whole book on the subject[1] (and parts of several other books on closely related topics). Other researchers on human intelligence give Keith E. Stanovich, the author of the submitted article, credit for bringing up findings from experimental studies on human thinking that will have be taken into account as cognitive psychology refines its theories on how you and I think. I see several comments in this thread along the lines of "The doesn't ponder" or "the author doesn't make the case" as if this article, restricted by a length limit, is the only writing he has ever produced on the topic. He mentions his longer book in the article. The article submitted today is a brief popular summary of an ongoing research program that the author has thought about as deeply as anyone on earth. [1] http://yalepress.yale.edu/yupbooks/book.asp?isbn=9780300164626 http://yalepress.yale.edu/yupbooks/book.asp?isbn=97803001646...
- unclebucknasty 12y ago
  Well, I personally think the current editorialized title here ("When rational thinking is correlated with intelligence the correlation is modest") is off, but the title of the article itself ("Rational and Irrational Thought: The Thinking that IQ Tests Miss") is more accurate. The article essentially points more to the disparity between IQ test results and rational thinking ability, which is more a critique of IQ testing. However, the title here points to a disparity between intelligence and rational thinking capacity, which is certainly counterintuitive. So, those who didn't note the difference in titles and/or went in with the idea that the title here was the author's premise may have taken issue with the article. The title editorializing was likely purposeful in that many people dismiss IQ tests out-of-hand, so the notion that IQ tests aren't predictive of rational-decision making capacity would have been far less remarkable or click-worthy.
deleted 12y ago
[deleted]
swatow 12y ago
I'm going to assume that IQ does measure general intelligence, since it is highly correlated with all sorts of positive life outcomes that seem to be associated with general intelligence. Given this, I'm more inclined to draw the conclusion that rationality is not central to intelligence. Most real life problems don't reduce to a logically complex puzzle. As my kung fu teacher once said "fighting isn't like a game of chess". In general, real life problems involve a mix of hard logical constraints, and soft constraints that cannot be reasoned about, but rather rely on intuition.
karmacondon 12y ago
The author doesn't make the case for why these types of reasoning skills are important, or why IQ is important for that matter. This isn't the first time that I've encountered brain teasers, and I know that I dislike them because I generally suck at them. The author characterizes this as being a "cognitive miser" (I also hate it when people make up terms), or not putting forth the effort to thoroughly think through the problem and just going with the easy answer. I can accept that this is what's happening, but what the hell does it mean? Should we be hiring people who are good at brain teasers? Does that kind of thinking indicate the ability to ship products, come up with novel solutions or understand things in a way that leads to significant increases to the bottom line? In my experience, everybody is some kind of stupid. Even puzzle masters occasionally lose their keys or say embarrassing things at parties. Intelligence and the lack there of is so fractured and interconnected that it doesn't seem like any kind of test is going to be able to measure it accurately, or even nail down what 'it' is. Some of the smartest people I know were good at questions like this, or maybe I just think they're smart because they're so good at these questions. Big companies love to use them during interviews and there's probably a reason for that. It's definitely a skill I would like to improve myself, but I still don't fully understand the value. Life isn't a series of puzzles, it's a series of problems punctuated by sheer randomness. Being able to take this or any kind of test can't be highly correlated with "success" by most measures, and certainly not in the ways that matter to most organizations.
- leoc 12y ago
  The article mentions one context where this kind of proper reasoning is highly important: doctors assessing the value of treatments for their patients, the usefulness of screening tests and the likely menaing of test results.
- anextio 12y ago
  The point is that these are not just brain teasers, these are just distilled examples of problems with which the brain often chooses a short-cut algorithm. The implication is that analogues of these situations happen all the time, and your ability to apply rational thought is proportional to the likelihood of achieving desirable outcomes.
erdevs 12y ago
I'm surprised the author doesn't ponder the following: Perhaps higher IQ means a person has more ability to solve these sorts of problems, if they take the time to exert the effort. That ability doesn't guarantee success in these sorts of questions without applied effort, however. That would mean IQ is still very, very important. But that it is not the only importans factor in solving problems well (eg rationally). To me, that seems quite plausible. Not sure why the author takes a rather anti-IQ slant as opposed for promoting supplemental measures.
- deleted 12y ago
  [deleted]
- aetherson 12y ago
  I'd like to split the difference and say that sure, IQ means that a person does have more ability to solve these sorts of problems -- but the fact that it doesn't actually prompt you to apply it is a serious deficit for IQ, and an under-recognized problem. Smart people get lots of things badly wrong lots of the time. The fact that they're smart can make others (and perhaps themselves, too) assume that they won't be badly wrong, and thus broaden the error.
- raverbashing 12y ago
  This It's one thing to solve an informal problem in a magazine, another one to sit and think about a certain problem, especially when you have hints about weather your guess is right or wrong or previous knowledge (the viral disease test is basically Bayes' Theorem)
- jquery 12y ago
  Exactly. It's rational to look at these "trick" questions, give the obvious answer, and quickly move on. What do you gain by sweating over them? It's not like they came up on the LSAT and correctly determining the answer determines your future. Flexing your IQ uses a lot of energy... literally.
stephen 12y ago
These books that are kind of pop-science descriptions of concepts like dysrationality are great, but what I'd love is a set of workooks (at various levels, e.g. my kids, adult learners like me, etc.) to systematically teach these skills. Without just being a list of brain teasers where you learn to recognize each trick. Do books like this exist? I'd love recommendations.
- Strilanc 12y ago
  CFAR (the Center for Applied Rationality)'s goal is to create workshops that effectively train people to be rational. Probably not what you want, though. They're pricey and not targeted at kids.
MarcScott 12y ago
"shallow processing can lead physicians to choose less effective medical treatments." But shallow processing can lead ER physicians to choose the correct medical treatment when time is of the essence. These type of multiple-choice problems annoy me, admittedly because I'm bad at them, but also because you don't always want to over analyze the problem and waste valuable hours when a solution that 'will do the job' is a few seconds of mental processing away.
- the_af 12y ago
  I don't think anyone disagrees (even the author) that this kind of "shortcut" reasoning is valuable. People want to spend as little energy as possible, and that makes a lot of sense! I'm not a biologist or a scientist, but this probably makes sense from an evolutionary perspective. The problem is when this kind of thinking leads to terrible results. One example would be thinking the effectiveness of a medical treatment is pretty good, when it's actually worse than not administering the treatment. And doctors are guilty of this, which in this case has serious consequences and is not a multiple-choice riddle!
  - tunesmith 12y ago
    The question is how to tell the difference ahead of time - when is a heuristic approach going to lead to a "good enough" answer versus a "counterintuitively damaging" answer. It would suck if the only way to know for sure is to avoid shortcut reasoning in all cases.
yodsanklai 12y ago
"Jack is looking at Anne, but Anne is looking at George. Jack is married, but George is not. Is a married person looking at an unmarried person?" On a side note, in constructive propositional logic, the answer is indeed C (http://en.wikipedia.org/wiki/Intuitionistic_logic http://en.wikipedia.org/wiki/Intuitionistic_logic). In this type of formalism, you have to provide a witness for every existential, or choice for every disjunction. In particular the law of excluded middle isn't an axiom. In order to prove the above proposition, you would need to be able to prove whether Anne is married, or unmarried.
- cscurmudgeon 12y ago
  Yes. But "background logic" (which classical first-order logic strives to model) is needed to prove meta theorems about any of the constructive logics. While the object-level proofs in some constructive/intuitionistic logic might do away with the law of excluded middle, the meta-theoretics proofs (proofs of theorems about the formalism) usually use them quite freely. My point is that there is no formalism that can ignore LEM at all levels (informal --> meta-theoretical--> object). The constructive logics are quite useful for things like program generation but I am not sure if we should use them as a model for human reasoning.
- enord 12y ago
  Was going to point this out as well. For a moment I almost resigned myself to be among the 80%, luckily I found a way to smug myself out of self-perceived mediocrity.
  - deleted 12y ago
    [deleted]
- darkmighty 12y ago
  I find baring the law of the excluded middle an unnecessarily restrictive basis for logic; but regardless, isn't this a simple (A \/ ~A) case? I mean, you may not be able to prove that Anne is married or prove that Anne is unmarried, but can't you prove by that Anne is either married or unmarried, by the definition of marriege? Moreover, I had the idea that Intuitionistic logic is advocated to curb with non-constructive proofs over non-finite sets rather than the simple case you mentioned.
MrQuincle 12y ago
The one with the probability of the disease makes me always feel like a robot. Okay, so I know, I've to check for the prior. (Bayesian) Statistics 101. The one with 'jack looking at anne' I glimsed already that 80% chose C, so I spend just a bit more time to consider going through the entire decision tree. It again makes me feel like a robot. :-) There is something very mechanical to these types of questions and I would wholeheartedly agree that this probably is a dimension that is not correlated with intelligence.
datashovel 12y ago
I think the Jack / Anne / George problem was not fully formed if they want someone to come to the conclusion that 'A' is the answer. For example, in the real world there are more "states" a person can be in than "married" or "not married". For example someone can be "separated", which IMO is not fully in the "married" or "not married" states. To formulate the question properly it needs to say "in this make believe world, all people are either married or not married". This leads me down the path to conclude 'A' is the answer. Not without this extra bit of context.
- nights192 12y ago
  Datashovel, man, I don't understand how being "separated" could be considered distinct from being married- it's a marital status. It's likely that they're not looking for that variable- 'married' is a binary value, why would one need to clarify?
  - datashovel 12y ago
    I don't claim to know all there is to know about marriage, or all the different laws / cultures of marriage in the world throughout the entire history of human civilization. But I guess I wasn't prepared to assume that all possible situations in the world (regardless of culture, continent, country, religion, etc...) can be boiled down precisely to either "married" or "not married" states of being. I guess I would ask: Why not include the extra context to the question?
- phaemon 12y ago
  I find it odd that there are several people who were tripped up by this, and then seem exclusively to concentrate on the whole "married/unmarried" thing. The answer, of course, is C, because Anne might be Jack's pet dog or something and therefore isn't a person at all. :-)
  - datashovel 12y ago
    Great point :)
- 12y ago
enord 12y ago
I get really defensive when my intelligence is on the line. I read this entire article with a combination of pretend aloofness and childlike fear of getting gotten by the gotcha-questions. Apparently, my ego plays into my dysrationalia (for better or worse).
- nabla9 12y ago
  Being defensive is very crippling. It's hard to concentrate, you may feel angry, inpatient or suddenly tired and mind wanders away. Reading speed and comprehension slows down. On the other hand, being aware and honest of your emotions and how they affect your thinking is important critical thinking skill. When you can realize your defensive attitude you can usually overcome it with effort. If I remember correctly, high IQ people have less cognitive biases that affect formal reasoning, but high IQ don't reduce biases that distort the ability to question one’s own judgment. Ability to reflect own decisions is correlated with dispositions like "active open-mindedness."
- halfcat 12y ago
  The research of Dr. Carol Dweck is interesting here [1]. It says that students who were praised for "being smart" developed this ego/defensiveness problem and subsequently began to underperform. While students praised for working hard did not, and ended up outperforming the "smart" group. The amazing thing about the research, to me, is that this effect was very prominent and quick, and it happened based on only one sentence being told to the students on one occasion. It did not take years of parents telling their child, "you are so smart". It took one teacher telling the child once, "you did so well, you must have worked hard", for the effect to take place. [1] http://nymag.com/news/features/27840/ http://nymag.com/news/features/27840/
ccvannorman 12y ago
This is why society develops bigger and bigger problems at scale: A few clever powerful actors can easily manipulate the masses by virtue of these built-in logical failings of most people, often to achieve their own evil or immoral goals.
lumberjack 12y ago
I don't understand no. 3. People are being self interested. Why is that irrational?
- jonahx 12y ago
  I think the idea is that they don't see the bias. And while hurting Germany's export business isn't a problem (from the pov of self-interest), you can imagine other situations where it would be a problem. To stick with example, say they really do believe these dangerous cars should be banned on American streets (and doing so really would save lives), but when they find out the car is their beloved Ford Explorer, they rationalize their caution away. Now the self-interest is getting them to make dangerous decisions, and they don't even realize that's what's happening.
interdrift 12y ago
Rational thinking isn't always linked to intelligence( as problem solving intelligence ) at least in my opinion . I know this might sound irational but oh well.
jrapdx3 12y ago
Most people I know probably think I'm smart enough, yet I still didn't get the right answers to all the test questions posed in the article. I started to wonder how come. One question giving me no trouble at all was item 5. It was obvious to me that the problem was one I often encounter, deciphering the ratios is a skill developed reading countless medical research reports. After a while it seems natural, it's about something salient in my universe. I'm guessing the item about marital status of individuals, one looking at another, doesn't align with the salience of usual experience. In normal life we have to be sensitive to the context, and in the question that context was deliberately stripped away. It reduces to a purely abstract notion whereas people, even intelligent ones, really want to know "is she married or not". That information is lacking hence the "C" response. Some of the other questions are kind of tricky unless one is familiar with the statistical domain they arrive from. Sure there's a logic involved, but learning how to think in those logical terms is unlikely to occur unless it's germane to the individual. If someone I cared about tested positive for the disease (#4), you bet I'd have good reason to figure out the odds. Otherwise, probably not. The motivational aspect is highly relevant and in many cases the "missing link" between intelligence and worse than expected performance.
cubetime 12y ago
See also: Do Rational People Exist? http://srconstantin.wordpress.com/2014/06/09/do-rationalists-exist/ http://srconstantin.wordpress.com/2014/06/09/do-rationalists...
deleted 12y ago
[deleted]
danieltillett 12y ago
I got all of them right except 6. My initial response was that you had to turn over all the cards. Thinking about why I chose this option I realised that I was not trusting the rule that a card must have a number on one side if it has a letter on the other side. My experience has made me highly suspicious that the model I have been given by someone is correct so I always like to test the assumptions of the model if possible.
- learnstats2 12y ago
  Indeed. This question indicates that we should trust one part of what is written, and only question another part. But both parts are written in the same voice, by the same authority. I like your answer. I got the question 'correct' (in part because I've seen the exact same question years ago) but I wonder if the results teach us more about the questions than the answers. In many human psychology experiments where humans are said to behave irrationally, I often think they are really behaving rationally when the context of the situation is taken into account.
- otis_inf 12y ago
  I had it wrong too, but mostly because I had an issue with the 8 card: IMHO it has to be turned over as it might have e.g. a 'V' on the other side, and that side also belongs to the card, but the answer explicitly states 8 isn't needed because apparently the other side isn't part of the card so the rule only applies to the sides you initially see. A bit of a trick question IMHO.
  - pedrosorio 12y ago
    You got that wrong. The rule is "vowel on one side implies even number on the other side". The only cards that don't follow this rule are the ones that have a vowel and an odd number. A card with an 8 will never disprove the rule regardless of the letter on the other side, that's why you don't need to turn it.
    - otis_inf 12y ago
      Yeah I got that now. I feel pretty stupid after I found that out... :)
Rapzid 12y ago
What if Anne is a dog and can't be married? It was never established that three PEOPLE were looking at each other. This is what we referred to in school as a "trick question". Very rarely present on standardized testing. We were mostly trained to zero in on the lack of information and then choose "C". The factors that play into who choose what are like more numerous than the author has considered; or at least presented.
halfcat 12y ago
A lot of this says, to me, that the human brain doesn't do well with concepts like statistics and probabilities. I think the far more interesting questions are why we do irrational things in the face of clear and overwhelming evidence. For instance, the knowledge of how to become physically fit and wealthy is public knowledge and freely available to anyone with internet access. I know I shouldn't eat that giant piece of cake right before I go to bed, but I do anyway, every night. I think the answer is, we are ruled by our chemical makeup, much more than any of us care to admit. The research discussed in "The Power of Habit" is the most interesting as far as ways to overcome our dependence on chemical/hormonal states ruling our lives.

120 points by lkrubner 12y ago

I consider myself smart and rational, so I was surprised that I stumbled on many of the test questions. I got this one wrong, though you would think, having been programming computers for 15 years, I surely should have gotten this one right: Jack is looking at Anne, but Anne is looking at George. Jack is married, but George is not. Is a married person looking at an unmarried person? A) Yes B) No C) Cannot be determined Oddly enough, I feel that if this had been presented to me as a bit of Java or Ruby code, then I would have gotten this one correct. It's the informality of English, and the assumptions I make when reading English, that tripped me up. Computer code engages the rational side of my brain in a way that English does not. And to follow up on a point made long ago by Jeff Atwood, I think this (that IQ and rationality are different) also explains the strange inability of some smart people to learn how to program computers. I'm thinking of the post here: http://blog.codinghorror.com/separating-programming-sheep-from-non-programming-goats/ http://blog.codinghorror.com/separating-programming-sheep-fr... Jeff Atwood discusses this quote: "All teachers of programming find that their results display a 'double hump'. It is as if there are two populations: those who can [program], and those who cannot [program], each with its own independent bell curve. Almost all research into programming teaching and learning have concentrated on teaching: change the language, change the application area, use an IDE and work on motivation. None of it works, and the double hump persists." And also this quote: "Despite the enormous changes which have taken place since electronic computing was invented in the 1950s, some things remain stubbornly the same. In particular, most people can't learn to program: between 30% and 60% of every university computer science department's intake fail the first programming course. Experienced teachers are weary but never oblivious of this fact; brighteyed beginners who believe that the old ones must have been doing it wrong learn the truth from bitter experience; and so it has been for almost two generations, ever since the subject began in the 1960s."

Rainymood 12y ago
SPOILER ALERT (ANSWER OF RIDDLE BELOW): I don't really see where you got stuck. Could you mind explaining me why/how/where you got stuck? I couldn't find a satisfying 'aha!' moment so I just kind of brute forced all the possibilities ..
- mbell 12y ago
  I got it wrong, not due to lack of logic but due to lack of spending the time to untangle the strange wording. I read it as "Jack is looking at Anne, but Anne is looking at _Jack_." and assumed it was some sort of trick question. If they were really interested in testing logic they would deliver the problem in a clear manor rather than masking it strange wording.
- lkrubner 12y ago
  SPOILER ALERT (ANSWER OF RIDDLE BELOW): Like most people, my first thought was c. And yet, as I said, I suspect that if this had been presented as a bit of Java or Ruby, I would have been unlikely to make that mistake. I read English with a laziness that is different from how I read computer code.
  - deleted 12y ago
    [deleted]
- gjm11 12y ago
  The failure mode isn't getting stuck, it's failing to notice that there's actual work to do. The way you get this one wrong is a thought process like this: "Well, obviously we only know about the first and third links in the chain, and we aren't told anything about the second one. So of course we can't tell whether there's a married person looking at an unmarried one because we have no information about any two directly 'connected' people." (But less explicit -- if you think about it in that much detail you're likely to think of brute-forcing the possibilities.) Or like this: "Well, let's see: A looks at B looks at C, A is married, C not. Does A->B have to be married->unmarried? No; B could be married. Does B->C have to be married->unmarried? No; B could be unmarried. So we're done." Once it occurs to you that you could just kind of brute force all the possibilities, you're going to get it right. The problem comes when you short-circuit the process. (And the reason why doing so isn't just stupid is that often this sort of "pruning" of mental effort is key to working things out efficiently. It just happens that in this case the prunability is illusory.)

pessimizer 12y ago

Only if you define intelligence as IQ is this true. It's more accurate to say that scoring high on IQ tests doesn't correlate very strongly with reasoning skills. Or, more interestingly, that scores on IQ tests correlate far more with factors like parental incomes or pleasure reading during childhood than they do with reasoning skills.

UhUhUhUh 12y ago

Most IQ tests have various scales and subscales, verbal, non-verbal, speed, memory etc... Aside from the few subtests that measure fund of knowledge, speed of reasoning, pattern extraction/recognition and ability to abstract problems define (loosely) what is being measured as "intelligence." The "G" Factor. Note that none of these categories strictly implies rational thinking. You can even make a point that scoring high on any of these categories will make you less likely to make rational decisions.

sysk 12y ago
<rant>It seems there's a lot of parallels to be made between IQ tests and programming language micro-benchmarks. Also, they seem to be equally useful: that is, not very except at the extremes. The corollary is that if you didn't pick your programming language based on a micro-benchmark, you probably shouldn't give too much weight to IQ when hiring.

deleted 12y ago

[deleted]

tokenadult 12y ago

The author of the article kindly submitted to open this thread has thought deeply about the title issue of the article and has written a whole book on the subject[1] (and parts of several other books on closely related topics). Other researchers on human intelligence give Keith E. Stanovich, the author of the submitted article, credit for bringing up findings from experimental studies on human thinking that will have be taken into account as cognitive psychology refines its theories on how you and I think. I see several comments in this thread along the lines of "The doesn't ponder" or "the author doesn't make the case" as if this article, restricted by a length limit, is the only writing he has ever produced on the topic. He mentions his longer book in the article. The article submitted today is a brief popular summary of an ongoing research program that the author has thought about as deeply as anyone on earth. [1] http://yalepress.yale.edu/yupbooks/book.asp?isbn=9780300164626 http://yalepress.yale.edu/yupbooks/book.asp?isbn=97803001646...

unclebucknasty 12y ago
Well, I personally think the current editorialized title here ("When rational thinking is correlated with intelligence the correlation is modest") is off, but the title of the article itself ("Rational and Irrational Thought: The Thinking that IQ Tests Miss") is more accurate. The article essentially points more to the disparity between IQ test results and rational thinking ability, which is more a critique of IQ testing. However, the title here points to a disparity between intelligence and rational thinking capacity, which is certainly counterintuitive. So, those who didn't note the difference in titles and/or went in with the idea that the title here was the author's premise may have taken issue with the article. The title editorializing was likely purposeful in that many people dismiss IQ tests out-of-hand, so the notion that IQ tests aren't predictive of rational-decision making capacity would have been far less remarkable or click-worthy.

deleted 12y ago

[deleted]

swatow 12y ago

I'm going to assume that IQ does measure general intelligence, since it is highly correlated with all sorts of positive life outcomes that seem to be associated with general intelligence. Given this, I'm more inclined to draw the conclusion that rationality is not central to intelligence. Most real life problems don't reduce to a logically complex puzzle. As my kung fu teacher once said "fighting isn't like a game of chess". In general, real life problems involve a mix of hard logical constraints, and soft constraints that cannot be reasoned about, but rather rely on intuition.

karmacondon 12y ago

The author doesn't make the case for why these types of reasoning skills are important, or why IQ is important for that matter. This isn't the first time that I've encountered brain teasers, and I know that I dislike them because I generally suck at them. The author characterizes this as being a "cognitive miser" (I also hate it when people make up terms), or not putting forth the effort to thoroughly think through the problem and just going with the easy answer. I can accept that this is what's happening, but what the hell does it mean? Should we be hiring people who are good at brain teasers? Does that kind of thinking indicate the ability to ship products, come up with novel solutions or understand things in a way that leads to significant increases to the bottom line? In my experience, everybody is some kind of stupid. Even puzzle masters occasionally lose their keys or say embarrassing things at parties. Intelligence and the lack there of is so fractured and interconnected that it doesn't seem like any kind of test is going to be able to measure it accurately, or even nail down what 'it' is. Some of the smartest people I know were good at questions like this, or maybe I just think they're smart because they're so good at these questions. Big companies love to use them during interviews and there's probably a reason for that. It's definitely a skill I would like to improve myself, but I still don't fully understand the value. Life isn't a series of puzzles, it's a series of problems punctuated by sheer randomness. Being able to take this or any kind of test can't be highly correlated with "success" by most measures, and certainly not in the ways that matter to most organizations.

leoc 12y ago
The article mentions one context where this kind of proper reasoning is highly important: doctors assessing the value of treatments for their patients, the usefulness of screening tests and the likely menaing of test results.
anextio 12y ago
The point is that these are not just brain teasers, these are just distilled examples of problems with which the brain often chooses a short-cut algorithm. The implication is that analogues of these situations happen all the time, and your ability to apply rational thought is proportional to the likelihood of achieving desirable outcomes.

erdevs 12y ago

I'm surprised the author doesn't ponder the following: Perhaps higher IQ means a person has more ability to solve these sorts of problems, if they take the time to exert the effort. That ability doesn't guarantee success in these sorts of questions without applied effort, however. That would mean IQ is still very, very important. But that it is not the only importans factor in solving problems well (eg rationally). To me, that seems quite plausible. Not sure why the author takes a rather anti-IQ slant as opposed for promoting supplemental measures.

deleted 12y ago
[deleted]
aetherson 12y ago
I'd like to split the difference and say that sure, IQ means that a person does have more ability to solve these sorts of problems -- but the fact that it doesn't actually prompt you to apply it is a serious deficit for IQ, and an under-recognized problem. Smart people get lots of things badly wrong lots of the time. The fact that they're smart can make others (and perhaps themselves, too) assume that they won't be badly wrong, and thus broaden the error.
raverbashing 12y ago
This It's one thing to solve an informal problem in a magazine, another one to sit and think about a certain problem, especially when you have hints about weather your guess is right or wrong or previous knowledge (the viral disease test is basically Bayes' Theorem)
jquery 12y ago
Exactly. It's rational to look at these "trick" questions, give the obvious answer, and quickly move on. What do you gain by sweating over them? It's not like they came up on the LSAT and correctly determining the answer determines your future. Flexing your IQ uses a lot of energy... literally.

stephen 12y ago

These books that are kind of pop-science descriptions of concepts like dysrationality are great, but what I'd love is a set of workooks (at various levels, e.g. my kids, adult learners like me, etc.) to systematically teach these skills. Without just being a list of brain teasers where you learn to recognize each trick. Do books like this exist? I'd love recommendations.

Strilanc 12y ago
CFAR (the Center for Applied Rationality)'s goal is to create workshops that effectively train people to be rational. Probably not what you want, though. They're pricey and not targeted at kids.

MarcScott 12y ago

"shallow processing can lead physicians to choose less effective medical treatments." But shallow processing can lead ER physicians to choose the correct medical treatment when time is of the essence. These type of multiple-choice problems annoy me, admittedly because I'm bad at them, but also because you don't always want to over analyze the problem and waste valuable hours when a solution that 'will do the job' is a few seconds of mental processing away.

the_af 12y ago
I don't think anyone disagrees (even the author) that this kind of "shortcut" reasoning is valuable. People want to spend as little energy as possible, and that makes a lot of sense! I'm not a biologist or a scientist, but this probably makes sense from an evolutionary perspective. The problem is when this kind of thinking leads to terrible results. One example would be thinking the effectiveness of a medical treatment is pretty good, when it's actually worse than not administering the treatment. And doctors are guilty of this, which in this case has serious consequences and is not a multiple-choice riddle!
- tunesmith 12y ago
  The question is how to tell the difference ahead of time - when is a heuristic approach going to lead to a "good enough" answer versus a "counterintuitively damaging" answer. It would suck if the only way to know for sure is to avoid shortcut reasoning in all cases.

yodsanklai 12y ago

"Jack is looking at Anne, but Anne is looking at George. Jack is married, but George is not. Is a married person looking at an unmarried person?" On a side note, in constructive propositional logic, the answer is indeed C (http://en.wikipedia.org/wiki/Intuitionistic_logic http://en.wikipedia.org/wiki/Intuitionistic_logic). In this type of formalism, you have to provide a witness for every existential, or choice for every disjunction. In particular the law of excluded middle isn't an axiom. In order to prove the above proposition, you would need to be able to prove whether Anne is married, or unmarried.

cscurmudgeon 12y ago
Yes. But "background logic" (which classical first-order logic strives to model) is needed to prove meta theorems about any of the constructive logics. While the object-level proofs in some constructive/intuitionistic logic might do away with the law of excluded middle, the meta-theoretics proofs (proofs of theorems about the formalism) usually use them quite freely. My point is that there is no formalism that can ignore LEM at all levels (informal --> meta-theoretical--> object). The constructive logics are quite useful for things like program generation but I am not sure if we should use them as a model for human reasoning.
enord 12y ago
Was going to point this out as well. For a moment I almost resigned myself to be among the 80%, luckily I found a way to smug myself out of self-perceived mediocrity.
- deleted 12y ago
  [deleted]
darkmighty 12y ago
I find baring the law of the excluded middle an unnecessarily restrictive basis for logic; but regardless, isn't this a simple (A \/ ~A) case? I mean, you may not be able to prove that Anne is married or prove that Anne is unmarried, but can't you prove by that Anne is either married or unmarried, by the definition of marriege? Moreover, I had the idea that Intuitionistic logic is advocated to curb with non-constructive proofs over non-finite sets rather than the simple case you mentioned.

MrQuincle 12y ago

The one with the probability of the disease makes me always feel like a robot. Okay, so I know, I've to check for the prior. (Bayesian) Statistics 101. The one with 'jack looking at anne' I glimsed already that 80% chose C, so I spend just a bit more time to consider going through the entire decision tree. It again makes me feel like a robot. :-) There is something very mechanical to these types of questions and I would wholeheartedly agree that this probably is a dimension that is not correlated with intelligence.

datashovel 12y ago

I think the Jack / Anne / George problem was not fully formed if they want someone to come to the conclusion that 'A' is the answer. For example, in the real world there are more "states" a person can be in than "married" or "not married". For example someone can be "separated", which IMO is not fully in the "married" or "not married" states. To formulate the question properly it needs to say "in this make believe world, all people are either married or not married". This leads me down the path to conclude 'A' is the answer. Not without this extra bit of context.

nights192 12y ago
Datashovel, man, I don't understand how being "separated" could be considered distinct from being married- it's a marital status. It's likely that they're not looking for that variable- 'married' is a binary value, why would one need to clarify?
- datashovel 12y ago
  I don't claim to know all there is to know about marriage, or all the different laws / cultures of marriage in the world throughout the entire history of human civilization. But I guess I wasn't prepared to assume that all possible situations in the world (regardless of culture, continent, country, religion, etc...) can be boiled down precisely to either "married" or "not married" states of being. I guess I would ask: Why not include the extra context to the question?
phaemon 12y ago
I find it odd that there are several people who were tripped up by this, and then seem exclusively to concentrate on the whole "married/unmarried" thing. The answer, of course, is C, because Anne might be Jack's pet dog or something and therefore isn't a person at all. :-)
- datashovel 12y ago
  Great point :)
12y ago

enord 12y ago

I get really defensive when my intelligence is on the line. I read this entire article with a combination of pretend aloofness and childlike fear of getting gotten by the gotcha-questions. Apparently, my ego plays into my dysrationalia (for better or worse).

nabla9 12y ago
Being defensive is very crippling. It's hard to concentrate, you may feel angry, inpatient or suddenly tired and mind wanders away. Reading speed and comprehension slows down. On the other hand, being aware and honest of your emotions and how they affect your thinking is important critical thinking skill. When you can realize your defensive attitude you can usually overcome it with effort. If I remember correctly, high IQ people have less cognitive biases that affect formal reasoning, but high IQ don't reduce biases that distort the ability to question one’s own judgment. Ability to reflect own decisions is correlated with dispositions like "active open-mindedness."
halfcat 12y ago
The research of Dr. Carol Dweck is interesting here [1]. It says that students who were praised for "being smart" developed this ego/defensiveness problem and subsequently began to underperform. While students praised for working hard did not, and ended up outperforming the "smart" group. The amazing thing about the research, to me, is that this effect was very prominent and quick, and it happened based on only one sentence being told to the students on one occasion. It did not take years of parents telling their child, "you are so smart". It took one teacher telling the child once, "you did so well, you must have worked hard", for the effect to take place. [1] http://nymag.com/news/features/27840/ http://nymag.com/news/features/27840/

ccvannorman 12y ago

This is why society develops bigger and bigger problems at scale: A few clever powerful actors can easily manipulate the masses by virtue of these built-in logical failings of most people, often to achieve their own evil or immoral goals.

lumberjack 12y ago

I don't understand no. 3. People are being self interested. Why is that irrational?

jonahx 12y ago
I think the idea is that they don't see the bias. And while hurting Germany's export business isn't a problem (from the pov of self-interest), you can imagine other situations where it would be a problem. To stick with example, say they really do believe these dangerous cars should be banned on American streets (and doing so really would save lives), but when they find out the car is their beloved Ford Explorer, they rationalize their caution away. Now the self-interest is getting them to make dangerous decisions, and they don't even realize that's what's happening.

interdrift 12y ago

Rational thinking isn't always linked to intelligence( as problem solving intelligence ) at least in my opinion . I know this might sound irational but oh well.

jrapdx3 12y ago

Most people I know probably think I'm smart enough, yet I still didn't get the right answers to all the test questions posed in the article. I started to wonder how come. One question giving me no trouble at all was item 5. It was obvious to me that the problem was one I often encounter, deciphering the ratios is a skill developed reading countless medical research reports. After a while it seems natural, it's about something salient in my universe. I'm guessing the item about marital status of individuals, one looking at another, doesn't align with the salience of usual experience. In normal life we have to be sensitive to the context, and in the question that context was deliberately stripped away. It reduces to a purely abstract notion whereas people, even intelligent ones, really want to know "is she married or not". That information is lacking hence the "C" response. Some of the other questions are kind of tricky unless one is familiar with the statistical domain they arrive from. Sure there's a logic involved, but learning how to think in those logical terms is unlikely to occur unless it's germane to the individual. If someone I cared about tested positive for the disease (#4), you bet I'd have good reason to figure out the odds. Otherwise, probably not. The motivational aspect is highly relevant and in many cases the "missing link" between intelligence and worse than expected performance.

cubetime 12y ago

See also: Do Rational People Exist? http://srconstantin.wordpress.com/2014/06/09/do-rationalists-exist/ http://srconstantin.wordpress.com/2014/06/09/do-rationalists...

deleted 12y ago

[deleted]

danieltillett 12y ago

I got all of them right except 6. My initial response was that you had to turn over all the cards. Thinking about why I chose this option I realised that I was not trusting the rule that a card must have a number on one side if it has a letter on the other side. My experience has made me highly suspicious that the model I have been given by someone is correct so I always like to test the assumptions of the model if possible.

learnstats2 12y ago
Indeed. This question indicates that we should trust one part of what is written, and only question another part. But both parts are written in the same voice, by the same authority. I like your answer. I got the question 'correct' (in part because I've seen the exact same question years ago) but I wonder if the results teach us more about the questions than the answers. In many human psychology experiments where humans are said to behave irrationally, I often think they are really behaving rationally when the context of the situation is taken into account.
otis_inf 12y ago
I had it wrong too, but mostly because I had an issue with the 8 card: IMHO it has to be turned over as it might have e.g. a 'V' on the other side, and that side also belongs to the card, but the answer explicitly states 8 isn't needed because apparently the other side isn't part of the card so the rule only applies to the sides you initially see. A bit of a trick question IMHO.
- pedrosorio 12y ago
  You got that wrong. The rule is "vowel on one side implies even number on the other side". The only cards that don't follow this rule are the ones that have a vowel and an odd number. A card with an 8 will never disprove the rule regardless of the letter on the other side, that's why you don't need to turn it.
  - otis_inf 12y ago
    Yeah I got that now. I feel pretty stupid after I found that out... :)

Rapzid 12y ago

What if Anne is a dog and can't be married? It was never established that three PEOPLE were looking at each other. This is what we referred to in school as a "trick question". Very rarely present on standardized testing. We were mostly trained to zero in on the lack of information and then choose "C". The factors that play into who choose what are like more numerous than the author has considered; or at least presented.

halfcat 12y ago

A lot of this says, to me, that the human brain doesn't do well with concepts like statistics and probabilities. I think the far more interesting questions are why we do irrational things in the face of clear and overwhelming evidence. For instance, the knowledge of how to become physically fit and wealthy is public knowledge and freely available to anyone with internet access. I know I shouldn't eat that giant piece of cake right before I go to bed, but I do anyway, every night. I think the answer is, we are ruled by our chemical makeup, much more than any of us care to admit. The research discussed in "The Power of Habit" is the most interesting as far as ways to overcome our dependence on chemical/hormonal states ruling our lives.