11 ms·

The quiz Daniel Kahneman wants you to fail

128 points by bootload 15y ago

taylorbuley 15y ago
My "choice & behavior" focus in college was pretty obscure at the time so it's cool to see Prospect Theory and other concepts start to go mainstream.
mixmax 15y ago
His nobel prize lecture is well worth a watch if you have 37 minutes to spare. http://www.nobelprize.org/mediaplayer/index.php?id=531 http://www.nobelprize.org/mediaplayer/index.php?id=531
- sesqu 15y ago
  This lecture revealed an error in question 5. The Vanity Fair article says "Kahneman and Tversky debunked Bernoulli’s utility theory, a cornerstone of economic thought since the 18th century. (Bernoulli first proponed that a person’s willingness to gamble a certain amount of money was a product of how that amount related to his overall wealth—that is, $1 million means more to a millionaire than it does to a billionaire.)" But in the lecture, Kahneman stated no qualms with utility theory. Rather, he pointed to the application of the theory, where Bernoulli had assumed a gain of $1 million is equivalent to $1 million appearing in their bank account, with the mechanism being irrelevant. That is what prospect theory (and the associated question) debunks.
nl 15y ago
And here's the reason why people who say economics isn't science are (mostly) wrong: economics is (mostly) applied statistics. Yes, interpretation of the statistics isn't "just science", but when is it ever?
- burgerbrain 15y ago
  Excuse my ignorance, but what does the scientific method have to do with statistics? I have always considered it to be a math, not science.
  - nl 15y ago
    I regard Math as a Science, although I do understand the controversy. http://en.wikipedia.org/wiki/Mathematics#Mathematics_as_science http://en.wikipedia.org/wiki/Mathematics#Mathematics_as_scie... is a pretty good summary. (Although if economics is Math then that is even better: economics is something that can make predictions, and isn't just the black magic that some seem to think) The scientific method applies is deciding what statistical experiments to do.
atuladhar 15y ago
I recently read Kahneman's 2011 book "Thinking, Fast and Slow" -- it should probably be required reading for everybody who's in charge of making decisions that affect a lot of people.
- jacques_chester 15y ago
  I have to second this advice, even though I'm halfway through it. Kahneman has spent decades investigating biases and cognitive error. And it turns out he's not all that bad at popularising his and related research. Of particular fascination (and frustration!) are the little examples he liberally sprinkles throughout the book. Small quizzes, questions and the like for the reader to try. Try as I might, I have consistently "failed" these in the way Kahneman goes on to explain. He even explains why he does that: because it's hard to particularise from the general. It's easy to generalise from the particular. I've found that many dimly held intuitions I've developed over the years about "how the world works" are starkly illustrated in the book. And many things I believed were not borne out be research. Bottom line: it's a great book and very illuminating. Read it.
  - nkurz 15y ago
    I'm halfway through it as well, but have a less positive impression. I came into it with a lot of respect for Kahneman and his research, but the first half of book is extremely flat. Much like the reaction here to this Vanity Fair teaser, I constantly find myself quibbling with the examples, and disagreeing with the explanations. I'm not sure who to blame, though. I think the book was written over a considerable period of time. Perhaps Kahneman's standards have changed? Perhaps there were multiple editors involved, some of whom (like the Vanity Fair author) didn't really understand the material? Or perhaps the editor was great, but only worked on some of the chapters. I'd offer a much less enthusiastic bottom line: It's a frustrating book, but you should read it anyway. Skip to Chapter 20 "The Illusion of Validity" if you get bogged down.
    - jacques_chester 15y ago
      I agree that the writing style is flat, but it's also very thorough and methodical. More than once I've found myself thinking "but what about X?" and then there, on the very next page, he addresses X to a depth I'd never considered. The biggest counterargument against his tests is that like a lot of sentences, they can be read in more than one way. Possibly for people with System 2 minds that like to dissect concepts, that's enormously annoying.
Uhhrrr 15y ago
The engineer/lawyer problem explanation (spoilers) is only correct in a world where lawyers and engineers have all the same characteristics. That the sampled individual is a man will skew things all by itself. Of the lawyers, approximately 40% will be women, whereas only 11% of the engineers. So our samplee could be one of 27 engineers or 42 lawyers - we've already bumped Peng from .3 to .39! That he likes math puzzles easily takes Peng to over .4, meaning the answer can't be A).
- nostromo 15y ago
  They didn't do a good job representing that particular heuristic. Here's a better example: http://en.wikipedia.org/wiki/Representativeness_heuristic#The_Taxicab_problem http://en.wikipedia.org/wiki/Representativeness_heuristic#Th... A better way to explain the common response to question #2 would be "quiz questions usually only give me information relavant to their answers" heuristic.
- deleted 15y ago
  [deleted]
- gburt 15y ago
  The author of this quiz seems to have completely misunderstood the relevant research. Either he has to assume that all individuals are identical (in which case, the little story is irrelevant) or he needs to apply Bayes rule according to the probabilities associated with the factors expressed in the personality exposition. Either way, the article's explanation for that particular question is wrong.
  - diego 15y ago
    Exactly. Here's the example used in Kahneman's book: "Dick is a 30-year-old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues. This description was intended to convey no information relevant to the question of whether Dick is an engineer or a lawyer." The description in the quiz is very different.
repsilat 15y ago
I'm afraid I don't quite understand the point of the second question. Does the answer mean to state authoritatively that that engineers are no more likely to do carpentry and partake in recreational mathematics than lawyers, or is there another explanation? If it were changed to something like "2. A team of psychologists performed personality tests on 100 professionals, of which 50 were engineers and 50 were nurses. Brief descriptions were written for each subject. The following is a sample of one of the resulting descriptions:" "'Jack is a 45-year old man.'" and if we were to assume that 80% of engineers are men, and that 80% of nurses are women, we'd expect 80% of the men to be engineers (and so expect Jack to be an engineer with 80% probability). Maybe? Question 4 - the theatre ticket - could be worded more clearly. I thought "as you enter the theatre" meant "as you enter the big room with the stage and lots of chairs". Who would buy a ticket after entering that room, no matter what the circumstance?
- SatvikBeri 15y ago
  The question is really wrong. I've heard a similar question before and it goes more like this: "A team of psychologists performed personality tests on 100 professionals, of which 5 were engineers and 95 were lawyers. Brief descriptions were written for each subject. The following is a randomly picked sample of one of the resulting descriptions: 'Jack is a 45-year old man who enjoys recreational Mathematics.' Assume 90% of engineers enjoy recreational Mathematics and only 10% of lawyers do. What is the chance that Jack is an engineer?" And the point is that the odds are still that Jack is a lawyer. The question as posed in the article, though, really doesn't make sense.
- inopinatus 15y ago
  I recently passed (with high distinctions) three postgraduate course exams towards my masters degree (at a top-10 university). Ho hum, don't be impressed; that is not unusual. My strategy, though, is not textbook; I spend more time studying the conceits of the lecturer, than I did the lecture material itself. With this in mind: The point we're supposed to accept is that when asked to evaluate the data sets presented, one heuristically makes assumptions based on social norms and crystallised intelligence. I'm pretty sure that this hypothesis was made in the article preceding the quiz, so no excuses for not realising the point of, well, any of the questions. For the theatre question, I too started with your interpretation. However, it didn't make sense in the context of illustrating the author's hypothesis, so it couldn't have been the "right" answer. Protip: when considering someone's line of argument in anything but a hard engineering discipline, I usually play the man before the ball - it's much more revealing.
jonathlee 15y ago
Despite the article's imputation that heuristics is a quick vs. accurate trade off, Gerd Gigerenzer's work show that in fact it is usually "quick AND more accurate" when used in the real, "large", world versus the "small" world of games and logical puzzles where all of the rules are known and knowledge of the problem is perfect, with infinite time allowed for optimization and calculation. The video here is well worth watching to counter-balance Kahneman's focus on edge cases where heuristic's break down or the wrong heuristic was applied. http://videolectures.net/ijcai2011_gigerenzer_heuristicus/ http://videolectures.net/ijcai2011_gigerenzer_heuristicus/
- aangjie 15y ago
  +1 for pointing out the contradictory research.....
Vivtek 15y ago
I didn't much like this: "being swayed by the way in which questions are worded rather than responding just to their substance" (for the lost ticket being allegedly equivalent to $10). Is a ticket, which costs $10, emotionally equivalent to $10? Once bought, the ticket is unique in my eyes, whereas I'm not even sure how many $10 bills I have in my wallet even now. So if I lose one, well, maybe it wasn't there in the first place. I guess what I'm trying to say is that the impact of the ticket loss and the impact of loss of a $10 bill are not equivalent in terms of substance. Unless you're an economist.
- yariang 15y ago
  I would dare say unless you're rational, not necessarily an economist. Most of us are not rational all the time, and the whole point of the question is to show an instance where we are not. You would be correct if there were not other tickets available; in that case, the ticket truly is unique. But in the example given you could acquire another one. In that case the ticket is not unique and is equivalent to the cost of acquiring another one, which is $10.
  - amireza 15y ago
    Is it rational to considers unknown possibilities and assigns gain to them and then decide, and if so what would be the numbers?
  - chrischen 15y ago
    But I believe it is the duty of the ticket vendor to keep track of who bought tickets. In protest, I would not replace a ticket if I lost it, but if I lost $10 it is completely my responsibility. Is this not rational if I believe I can affect change through the action?
    - yariang 15y ago
      That's an entirely different issue than the one I was responding to. In that case, it would be rational if your actions could reasonably affect change, which in this case I don't think they would. First, I don't see an obvious need for theaters to keep track of who purchased tickets--save the case of online transactions obviously. Granted, the whole example is contrived--but I don't see the utmost need for it. And, even if they had a moral imperative to do so and didn't, this would mostly affect people--by definition--who purchased a ticket already, and either lost it or want to return it or something of that nature. It is safe to assume few people lose their tickets and of those that do, not all of them protest by not buying another one. Protests are usually only effective if they hurt the company in terms of reputation or money. Since that is unlikely (given the vague probabilities I mentioned and specially because you already paid for one ticket), your protest will most likely be in vain. We could of course get into a philosophical debate over the worth of such protest, but this is neither the time or place. Protesting the theater record-keeping policies in the way you mention and for the reasons you mention would probably be, in my opinion and with all the information you provided, rationally irrational.
jpiasetz 15y ago
I don't understand the title? Why does it matter either way if someone fails or gets it right?
- yariang 15y ago
  If most people answered all or most of the questions correctly, it would be disproving Kahneman and Tverksy's theory of how people think. The title assumes they would hate to be proven wrong, but makes sense.
jimrandomh 15y ago
Kahneman's work is great, and deserves to have attention directed to it, but this Vanity Fair article is pretty bad. Why does the title say that Kahneman wants people to fail the quiz? That's wrong, and since it's attention-grabbing in a way that the truth isn't, I'd say it's dishonest. And the rest of it is riddled with errors and confusions. So go read Daniel Kahneman's books, and don't read Vanity Fair.
colanderman 15y ago
The questions involving "90% chance of $1000 or 100% chance of $900" always bother me. I never understand why economists think that a rational actor would consider them equivalent; they're not, unless you are making that choice many many times. But if I'm given that chance once (which is presumably what most participants assume, since that's not a choice that comes up often in one's life), it's really then a choice between "90% of a ton of free money or 100% chance of a ton of free money". Unless the dollar amounts are radically different, who in their right mind would take the choice that could possibly leave them without a life-changing sum the next day?
- 2arrs2ells 15y ago
  The interesting thing is not that people are risk averse (and thus choose 100% chance of $900), the interesting thing is that people become risk seeking when it comes to losses (and thus choose 90% chance of -$1000). You're creating a bit of a straw man when it comes to behavioral economists and their view of "rational actors" - the whole field is built around the understanding that there is more to economic decisions than expected value.
  - X-Istence 15y ago
    Yes, however there is a slight chance of winning the odds and not paying anything. The risk is minimal. If the numbers were further apart I don't think the same logic would kick in. A = $900 lost B = $9000 lost with a 90% chance Those are odds I wouldn't want to try. I would take the instant loss knowing that my odds are just not good enough to win if I were to choose B.
    - anothermachine 15y ago
      you mean 10% chance, right?
    - neonkiwi 15y ago
      The reason that $1000 at 90% odds and $900 at 100% odds are used in this example is the expected value is the same in both cases, making the situations 'equivalent'. A 90% chance of losing $9000 has an expected value of -$8100.
sdizdar 15y ago
The paper they wrote in 1983 is a must read for all entrepreneurs (and sales people in general): http://pol-sci.uga.edu/courses/2008/ainsworthfall08kahnemantversky.pdf http://pol-sci.uga.edu/courses/2008/ainsworthfall08kahnemant...
usaar333 15y ago
As someone who has had lots of economics/probability training, this quiz really doesn't do the research justice. It gets a point across, but could have been presented better. I was bothered by its misuse of terminology and the quiz not really being one. Spoilers ahead! 1. The answer could be A or C, depending on how small and large the hospitals are. The small hospital could easily be expected to have < 5% more variance (depending on definition of "small" and what 5% of each other means) depending on its and the large one's size. 2. I don't see why the correct response must be 30%. Such attributes conditionally describe an engineer better than a lawyer. Assuming random sampling of lawyers and engineers, I'd be surprised if the answer isn't > 30%. I'm not sure what the numbers are actually, as I don't know how strong this conditional applies to engineers or lawyers. The point of the research is that people over-emphasize the conditional over the prior (indeed often ignore the prior), not that people should not use conditional information. 3. " it is likely that your answer to question (a) is positively correlated to your answer to question (b)" I understand what they are trying to say, but the wording is quite off. Correlation is a property of data, not an individual point. If I'm the only person who ever takes this test, my answers have undefined correlation. The modifier "likely" is especially baffling (correlation is a constant property of data). Proper (and less verbose!) terminology is "people's answers to (a) are positively correlated to (b)"
janj 15y ago
I've started reading 'Thinking, Fast and Slow' and really like it so far. If anyone in the Bay area wants it after I'm done let me know.
tikhonj 15y ago
Not exactly the same, but my favorite visceral examples of imperfect heuristics are "garden-path sentences"[1]. These are sentences that trick you into having to backtrack when you're parsing them, which brings the whole parsing process into sharp relief. [1]: http://en.wikipedia.org/wiki/Garden_path_sentence http://en.wikipedia.org/wiki/Garden_path_sentence A good example is: The old man the boat. Figure out why this sentence is actually grammatically valid and then read the article.
- SilasX 15y ago
  To save everyone the trouble of reading a lot to learn a little: "man" is a verb in that sentence. Though if I wanted to be clever I could hammer that into less garden-pathy sentence as follows: "In my will, I left everything to the one who could use it best. I gave the boy the legos. I gave the girl the dollhouse. I gave the mother the kitchen set. The father the hunting rifle. The bachelor the suit. The old man the boat."
rayiner 15y ago
A good summary paper on behavioral economics by a professor at Caltech: http://www.hss.caltech.edu/~camerer/ribe239.pdf http://www.hss.caltech.edu/~camerer/ribe239.pdf Dan Kahneman, Richard Thaler and Dan Ariely have both published some popular books on the subject. Thinking Fast, Thinking Slow just came out last year. Predictably Irrational came out a few years ago. Nudge (Cass Sunstein and Richard Thaler) looks at the implications of behavioral economics for the law. There is a good summary article of that work here: http://www.law.harvard.edu/programs/olin_center/papers/pdf/236.pdf http://www.law.harvard.edu/programs/olin_center/papers/pdf/2.... I personally think this work is pretty earth-shattering in the field, and that the above work is a must-read for anyone interested in economics. The engineer side of me is really attracted to the fact that behavioral economics uses legitimate experimental methodology, instead of mathematically-supported handwaving. And the implications of the work really turn some of our assumptions about the nature of the economic system on their head.
- NeilCJames 15y ago
  Much earlier ('91, I think), but a better read for those with a technical mindset (but still accessible) is Thaler's The Winner's Curse.
joshu 15y ago
I also recommend reading Fortune's Formula. I took a stab at Prospect Theory myself: http://joshua.schachter.org/2008/09/amateur-economist.html http://joshua.schachter.org/2008/09/amateur-economist.html Took a few hours of work.
DannyPage 15y ago
"The framing effect is also used to explain the influence of positive and negative information on our decisions—for example, why consumers prefer to buy ground beef labeled 80 percent lean rather than 20 percent fat." Then why don't politicians say "Employment rates are at 91%" instead of the depressing "Unemployment rates are at 9%"?
kia 15y ago
Question three about dates is pretty much Anchoring Effect: http://en.wikipedia.org/wiki/Anchoring http://en.wikipedia.org/wiki/Anchoring Also an interesting reference about anchoring and electric cars from the above Wikipedia article: http://www.treehugger.com/cars/nissans-leaf-creates-a-new-anchor-for-electric-car-pricing-forcing-others-to-match-it.html http://www.treehugger.com/cars/nissans-leaf-creates-a-new-an...
- icefox 15y ago
  The thing is question three is about dates and question three B is about how happy you are so the two are suppose to be related. If it was question three and question four maybe, but not when they put the two questions together.
paganel 15y ago
Sorry to say it, but this certainly smells like cargo-cult science. First, I'm not a scientist, and not even a Nobel-prize winner, but for example I cannot understand what question 3) has to do with "science". I did have 2 dates in the past month, but I'm at one of the lowest points of my life, because, guess what? the missing context really matters (I'm about to get divorced). The same goes for question 4) . I chose A, "Yes", assuming it was a good movie worth seeing, but lacking the above-mentioned "context" (is it "Forrest Gump" or "Delta Force IV"?) any answer you give it's just to satisfy the interviewer. And yes, people DO go and see movies that they know they're bad. A little less reading on "behavioral economics" or "animal spirits" and a little more reading of Hume will do everyone a ton of good.
- nl 15y ago
  The questions aren't science - that statistics that predict that most people will get them wrong are. Or rather - the theories which predict how people will (statistically) make sub-optimal decisions in certain circumstances are science: they make predictions that can be tested.
  - Radim 15y ago
    I believe paganel is referring exactly to the "sub-optimal decision" part, in the explanation (not the questions themselves). And yes, this quiz seems a load of underspecified cargo-cult horsecrap. I surely hope the original research was of higher quality.
    - nl 15y ago
      I believe paganel is referring exactly to the "sub-optimal decision" part, in the explanation (not the questions themselves). Can you expand on that? The point of the article is that people will (statistically) get the questions wrong, or at least have biases in their responses. For example, in the first question the grandparent poster may well have had good reasons to answer as they did, but that doesn't discount the fact that most people are vulnerable to the attribute substitution bias[1]. And yes, this quiz seems a load of underspecified cargo-cult horsecrap. That's to be expected isn't it? They are trying to show five different statistical effects in five questions. To do it properly they would have to ask 10's of questions for each one, but I don't think that would work for a Vanity Fair sidebar article. Instead, they tried to find questions that would demonstrate the principles to most people. [1] http://en.wikipedia.org/wiki/Attribute_substitution http://en.wikipedia.org/wiki/Attribute_substitution
gizzlon 15y ago
The ticket questions is dumb. Has anyone heard a better version of it? At least to me, the important factors are: Do I remember where I'm suppose to sit? Would It be immoral to attend even if I lost the physical stub?
srl 15y ago
After the first question: after being primed not to trust my instincts, I decided (SPOILER) it couldn't be (a), and thus guessed (c). Which I suppose reflects another important cognitive bias.
berdario 15y ago
What does "How many dates did you have last month?" even means? Maybe I've a different notion of "date" from the rest of the people... but I'd assume that if you've a stable relationship with a significant other, then your number could be definitely higher than 5 in a month... otherwise, if it's considering all the different dates with all potential partners, 3 to 5 seems an number exceptionally high (in fact: I've got a grand total of 0 dates over the last 24 months, and I'm quite sad about that)
kstenerud 15y ago
Their interpretation of question 5 is just plain wrong. Aside from the fact that losses DO hurt more than gains (because you need to alter your plans to accomodate losses), the assumption of equivalence is fundamentally flawed. The problem is that the two halves are inverted. Correcting: 5a: Your base position is $900, and you have the option of risking $900 for a possible gain of $100. You have a 90% chance of gaining $100 (a trivial amount) and a 10% chance of losing $900 (a big amount). 5b: Your base position is -$900, and you have the option of risking $100 for a possible gain of $900. You have a 90% chance of losing $100 (a trivial amount) and a 10% chance of gaining $900 (a big amount). Since you're an individual, and not a statistical average, there's no middle ground in the values gained or lost. So the rational person is right to choose A, then B. The two are NOT equivalent to an individual. The trivial amount makes little difference as a gain or a loss, but the big amount does.

128 points by bootload 15y ago

My "choice & behavior" focus in college was pretty obscure at the time so it's cool to see Prospect Theory and other concepts start to go mainstream.

mixmax 15y ago

His nobel prize lecture is well worth a watch if you have 37 minutes to spare. http://www.nobelprize.org/mediaplayer/index.php?id=531 http://www.nobelprize.org/mediaplayer/index.php?id=531

sesqu 15y ago
This lecture revealed an error in question 5. The Vanity Fair article says "Kahneman and Tversky debunked Bernoulli’s utility theory, a cornerstone of economic thought since the 18th century. (Bernoulli first proponed that a person’s willingness to gamble a certain amount of money was a product of how that amount related to his overall wealth—that is, $1 million means more to a millionaire than it does to a billionaire.)" But in the lecture, Kahneman stated no qualms with utility theory. Rather, he pointed to the application of the theory, where Bernoulli had assumed a gain of $1 million is equivalent to $1 million appearing in their bank account, with the mechanism being irrelevant. That is what prospect theory (and the associated question) debunks.

nl 15y ago

And here's the reason why people who say economics isn't science are (mostly) wrong: economics is (mostly) applied statistics. Yes, interpretation of the statistics isn't "just science", but when is it ever?

burgerbrain 15y ago
Excuse my ignorance, but what does the scientific method have to do with statistics? I have always considered it to be a math, not science.
- nl 15y ago
  I regard Math as a Science, although I do understand the controversy. http://en.wikipedia.org/wiki/Mathematics#Mathematics_as_science http://en.wikipedia.org/wiki/Mathematics#Mathematics_as_scie... is a pretty good summary. (Although if economics is Math then that is even better: economics is something that can make predictions, and isn't just the black magic that some seem to think) The scientific method applies is deciding what statistical experiments to do.

atuladhar 15y ago

I recently read Kahneman's 2011 book "Thinking, Fast and Slow" -- it should probably be required reading for everybody who's in charge of making decisions that affect a lot of people.

jacques_chester 15y ago
I have to second this advice, even though I'm halfway through it. Kahneman has spent decades investigating biases and cognitive error. And it turns out he's not all that bad at popularising his and related research. Of particular fascination (and frustration!) are the little examples he liberally sprinkles throughout the book. Small quizzes, questions and the like for the reader to try. Try as I might, I have consistently "failed" these in the way Kahneman goes on to explain. He even explains why he does that: because it's hard to particularise from the general. It's easy to generalise from the particular. I've found that many dimly held intuitions I've developed over the years about "how the world works" are starkly illustrated in the book. And many things I believed were not borne out be research. Bottom line: it's a great book and very illuminating. Read it.
- nkurz 15y ago
  I'm halfway through it as well, but have a less positive impression. I came into it with a lot of respect for Kahneman and his research, but the first half of book is extremely flat. Much like the reaction here to this Vanity Fair teaser, I constantly find myself quibbling with the examples, and disagreeing with the explanations. I'm not sure who to blame, though. I think the book was written over a considerable period of time. Perhaps Kahneman's standards have changed? Perhaps there were multiple editors involved, some of whom (like the Vanity Fair author) didn't really understand the material? Or perhaps the editor was great, but only worked on some of the chapters. I'd offer a much less enthusiastic bottom line: It's a frustrating book, but you should read it anyway. Skip to Chapter 20 "The Illusion of Validity" if you get bogged down.
  - jacques_chester 15y ago
    I agree that the writing style is flat, but it's also very thorough and methodical. More than once I've found myself thinking "but what about X?" and then there, on the very next page, he addresses X to a depth I'd never considered. The biggest counterargument against his tests is that like a lot of sentences, they can be read in more than one way. Possibly for people with System 2 minds that like to dissect concepts, that's enormously annoying.

Uhhrrr 15y ago

The engineer/lawyer problem explanation (spoilers) is only correct in a world where lawyers and engineers have all the same characteristics. That the sampled individual is a man will skew things all by itself. Of the lawyers, approximately 40% will be women, whereas only 11% of the engineers. So our samplee could be one of 27 engineers or 42 lawyers - we've already bumped Peng from .3 to .39! That he likes math puzzles easily takes Peng to over .4, meaning the answer can't be A).

nostromo 15y ago
They didn't do a good job representing that particular heuristic. Here's a better example: http://en.wikipedia.org/wiki/Representativeness_heuristic#The_Taxicab_problem http://en.wikipedia.org/wiki/Representativeness_heuristic#Th... A better way to explain the common response to question #2 would be "quiz questions usually only give me information relavant to their answers" heuristic.
deleted 15y ago
[deleted]
gburt 15y ago
The author of this quiz seems to have completely misunderstood the relevant research. Either he has to assume that all individuals are identical (in which case, the little story is irrelevant) or he needs to apply Bayes rule according to the probabilities associated with the factors expressed in the personality exposition. Either way, the article's explanation for that particular question is wrong.
- diego 15y ago
  Exactly. Here's the example used in Kahneman's book: "Dick is a 30-year-old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues. This description was intended to convey no information relevant to the question of whether Dick is an engineer or a lawyer." The description in the quiz is very different.

repsilat 15y ago

I'm afraid I don't quite understand the point of the second question. Does the answer mean to state authoritatively that that engineers are no more likely to do carpentry and partake in recreational mathematics than lawyers, or is there another explanation? If it were changed to something like "2. A team of psychologists performed personality tests on 100 professionals, of which 50 were engineers and 50 were nurses. Brief descriptions were written for each subject. The following is a sample of one of the resulting descriptions:" "'Jack is a 45-year old man.'" and if we were to assume that 80% of engineers are men, and that 80% of nurses are women, we'd expect 80% of the men to be engineers (and so expect Jack to be an engineer with 80% probability). Maybe? Question 4 - the theatre ticket - could be worded more clearly. I thought "as you enter the theatre" meant "as you enter the big room with the stage and lots of chairs". Who would buy a ticket after entering that room, no matter what the circumstance?

SatvikBeri 15y ago
The question is really wrong. I've heard a similar question before and it goes more like this: "A team of psychologists performed personality tests on 100 professionals, of which 5 were engineers and 95 were lawyers. Brief descriptions were written for each subject. The following is a randomly picked sample of one of the resulting descriptions: 'Jack is a 45-year old man who enjoys recreational Mathematics.' Assume 90% of engineers enjoy recreational Mathematics and only 10% of lawyers do. What is the chance that Jack is an engineer?" And the point is that the odds are still that Jack is a lawyer. The question as posed in the article, though, really doesn't make sense.
inopinatus 15y ago
I recently passed (with high distinctions) three postgraduate course exams towards my masters degree (at a top-10 university). Ho hum, don't be impressed; that is not unusual. My strategy, though, is not textbook; I spend more time studying the conceits of the lecturer, than I did the lecture material itself. With this in mind: The point we're supposed to accept is that when asked to evaluate the data sets presented, one heuristically makes assumptions based on social norms and crystallised intelligence. I'm pretty sure that this hypothesis was made in the article preceding the quiz, so no excuses for not realising the point of, well, any of the questions. For the theatre question, I too started with your interpretation. However, it didn't make sense in the context of illustrating the author's hypothesis, so it couldn't have been the "right" answer. Protip: when considering someone's line of argument in anything but a hard engineering discipline, I usually play the man before the ball - it's much more revealing.

jonathlee 15y ago

Despite the article's imputation that heuristics is a quick vs. accurate trade off, Gerd Gigerenzer's work show that in fact it is usually "quick AND more accurate" when used in the real, "large", world versus the "small" world of games and logical puzzles where all of the rules are known and knowledge of the problem is perfect, with infinite time allowed for optimization and calculation. The video here is well worth watching to counter-balance Kahneman's focus on edge cases where heuristic's break down or the wrong heuristic was applied. http://videolectures.net/ijcai2011_gigerenzer_heuristicus/ http://videolectures.net/ijcai2011_gigerenzer_heuristicus/

aangjie 15y ago
+1 for pointing out the contradictory research.....

Vivtek 15y ago

I didn't much like this: "being swayed by the way in which questions are worded rather than responding just to their substance" (for the lost ticket being allegedly equivalent to $10). Is a ticket, which costs $10, emotionally equivalent to $10? Once bought, the ticket is unique in my eyes, whereas I'm not even sure how many $10 bills I have in my wallet even now. So if I lose one, well, maybe it wasn't there in the first place. I guess what I'm trying to say is that the impact of the ticket loss and the impact of loss of a $10 bill are not equivalent in terms of substance. Unless you're an economist.

yariang 15y ago
I would dare say unless you're rational, not necessarily an economist. Most of us are not rational all the time, and the whole point of the question is to show an instance where we are not. You would be correct if there were not other tickets available; in that case, the ticket truly is unique. But in the example given you could acquire another one. In that case the ticket is not unique and is equivalent to the cost of acquiring another one, which is $10.
- amireza 15y ago
  Is it rational to considers unknown possibilities and assigns gain to them and then decide, and if so what would be the numbers?
- chrischen 15y ago
  But I believe it is the duty of the ticket vendor to keep track of who bought tickets. In protest, I would not replace a ticket if I lost it, but if I lost $10 it is completely my responsibility. Is this not rational if I believe I can affect change through the action?
  - yariang 15y ago
    That's an entirely different issue than the one I was responding to. In that case, it would be rational if your actions could reasonably affect change, which in this case I don't think they would. First, I don't see an obvious need for theaters to keep track of who purchased tickets--save the case of online transactions obviously. Granted, the whole example is contrived--but I don't see the utmost need for it. And, even if they had a moral imperative to do so and didn't, this would mostly affect people--by definition--who purchased a ticket already, and either lost it or want to return it or something of that nature. It is safe to assume few people lose their tickets and of those that do, not all of them protest by not buying another one. Protests are usually only effective if they hurt the company in terms of reputation or money. Since that is unlikely (given the vague probabilities I mentioned and specially because you already paid for one ticket), your protest will most likely be in vain. We could of course get into a philosophical debate over the worth of such protest, but this is neither the time or place. Protesting the theater record-keeping policies in the way you mention and for the reasons you mention would probably be, in my opinion and with all the information you provided, rationally irrational.

jpiasetz 15y ago

I don't understand the title? Why does it matter either way if someone fails or gets it right?

yariang 15y ago
If most people answered all or most of the questions correctly, it would be disproving Kahneman and Tverksy's theory of how people think. The title assumes they would hate to be proven wrong, but makes sense.

jimrandomh 15y ago

Kahneman's work is great, and deserves to have attention directed to it, but this Vanity Fair article is pretty bad. Why does the title say that Kahneman wants people to fail the quiz? That's wrong, and since it's attention-grabbing in a way that the truth isn't, I'd say it's dishonest. And the rest of it is riddled with errors and confusions. So go read Daniel Kahneman's books, and don't read Vanity Fair.

colanderman 15y ago

The questions involving "90% chance of $1000 or 100% chance of $900" always bother me. I never understand why economists think that a rational actor would consider them equivalent; they're not, unless you are making that choice many many times. But if I'm given that chance once (which is presumably what most participants assume, since that's not a choice that comes up often in one's life), it's really then a choice between "90% of a ton of free money or 100% chance of a ton of free money". Unless the dollar amounts are radically different, who in their right mind would take the choice that could possibly leave them without a life-changing sum the next day?

2arrs2ells 15y ago
The interesting thing is not that people are risk averse (and thus choose 100% chance of $900), the interesting thing is that people become risk seeking when it comes to losses (and thus choose 90% chance of -$1000). You're creating a bit of a straw man when it comes to behavioral economists and their view of "rational actors" - the whole field is built around the understanding that there is more to economic decisions than expected value.
- X-Istence 15y ago
  Yes, however there is a slight chance of winning the odds and not paying anything. The risk is minimal. If the numbers were further apart I don't think the same logic would kick in. A = $900 lost B = $9000 lost with a 90% chance Those are odds I wouldn't want to try. I would take the instant loss knowing that my odds are just not good enough to win if I were to choose B.
  - anothermachine 15y ago
    you mean 10% chance, right?
  - neonkiwi 15y ago
    The reason that $1000 at 90% odds and $900 at 100% odds are used in this example is the expected value is the same in both cases, making the situations 'equivalent'. A 90% chance of losing $9000 has an expected value of -$8100.

sdizdar 15y ago

The paper they wrote in 1983 is a must read for all entrepreneurs (and sales people in general): http://pol-sci.uga.edu/courses/2008/ainsworthfall08kahnemantversky.pdf http://pol-sci.uga.edu/courses/2008/ainsworthfall08kahnemant...

usaar333 15y ago

As someone who has had lots of economics/probability training, this quiz really doesn't do the research justice. It gets a point across, but could have been presented better. I was bothered by its misuse of terminology and the quiz not really being one. Spoilers ahead! 1. The answer could be A or C, depending on how small and large the hospitals are. The small hospital could easily be expected to have < 5% more variance (depending on definition of "small" and what 5% of each other means) depending on its and the large one's size. 2. I don't see why the correct response must be 30%. Such attributes conditionally describe an engineer better than a lawyer. Assuming random sampling of lawyers and engineers, I'd be surprised if the answer isn't > 30%. I'm not sure what the numbers are actually, as I don't know how strong this conditional applies to engineers or lawyers. The point of the research is that people over-emphasize the conditional over the prior (indeed often ignore the prior), not that people should not use conditional information. 3. " it is likely that your answer to question (a) is positively correlated to your answer to question (b)" I understand what they are trying to say, but the wording is quite off. Correlation is a property of data, not an individual point. If I'm the only person who ever takes this test, my answers have undefined correlation. The modifier "likely" is especially baffling (correlation is a constant property of data). Proper (and less verbose!) terminology is "people's answers to (a) are positively correlated to (b)"

janj 15y ago

I've started reading 'Thinking, Fast and Slow' and really like it so far. If anyone in the Bay area wants it after I'm done let me know.

tikhonj 15y ago

Not exactly the same, but my favorite visceral examples of imperfect heuristics are "garden-path sentences"[1]. These are sentences that trick you into having to backtrack when you're parsing them, which brings the whole parsing process into sharp relief. [1]: http://en.wikipedia.org/wiki/Garden_path_sentence http://en.wikipedia.org/wiki/Garden_path_sentence A good example is: The old man the boat. Figure out why this sentence is actually grammatically valid and then read the article.

SilasX 15y ago
To save everyone the trouble of reading a lot to learn a little: "man" is a verb in that sentence. Though if I wanted to be clever I could hammer that into less garden-pathy sentence as follows: "In my will, I left everything to the one who could use it best. I gave the boy the legos. I gave the girl the dollhouse. I gave the mother the kitchen set. The father the hunting rifle. The bachelor the suit. The old man the boat."

rayiner 15y ago

A good summary paper on behavioral economics by a professor at Caltech: http://www.hss.caltech.edu/~camerer/ribe239.pdf http://www.hss.caltech.edu/~camerer/ribe239.pdf Dan Kahneman, Richard Thaler and Dan Ariely have both published some popular books on the subject. Thinking Fast, Thinking Slow just came out last year. Predictably Irrational came out a few years ago. Nudge (Cass Sunstein and Richard Thaler) looks at the implications of behavioral economics for the law. There is a good summary article of that work here: http://www.law.harvard.edu/programs/olin_center/papers/pdf/236.pdf http://www.law.harvard.edu/programs/olin_center/papers/pdf/2.... I personally think this work is pretty earth-shattering in the field, and that the above work is a must-read for anyone interested in economics. The engineer side of me is really attracted to the fact that behavioral economics uses legitimate experimental methodology, instead of mathematically-supported handwaving. And the implications of the work really turn some of our assumptions about the nature of the economic system on their head.

NeilCJames 15y ago
Much earlier ('91, I think), but a better read for those with a technical mindset (but still accessible) is Thaler's The Winner's Curse.

joshu 15y ago

I also recommend reading Fortune's Formula. I took a stab at Prospect Theory myself: http://joshua.schachter.org/2008/09/amateur-economist.html http://joshua.schachter.org/2008/09/amateur-economist.html Took a few hours of work.

DannyPage 15y ago

"The framing effect is also used to explain the influence of positive and negative information on our decisions—for example, why consumers prefer to buy ground beef labeled 80 percent lean rather than 20 percent fat." Then why don't politicians say "Employment rates are at 91%" instead of the depressing "Unemployment rates are at 9%"?

kia 15y ago

Question three about dates is pretty much Anchoring Effect: http://en.wikipedia.org/wiki/Anchoring http://en.wikipedia.org/wiki/Anchoring Also an interesting reference about anchoring and electric cars from the above Wikipedia article: http://www.treehugger.com/cars/nissans-leaf-creates-a-new-anchor-for-electric-car-pricing-forcing-others-to-match-it.html http://www.treehugger.com/cars/nissans-leaf-creates-a-new-an...

icefox 15y ago
The thing is question three is about dates and question three B is about how happy you are so the two are suppose to be related. If it was question three and question four maybe, but not when they put the two questions together.

paganel 15y ago

Sorry to say it, but this certainly smells like cargo-cult science. First, I'm not a scientist, and not even a Nobel-prize winner, but for example I cannot understand what question 3) has to do with "science". I did have 2 dates in the past month, but I'm at one of the lowest points of my life, because, guess what? the missing context really matters (I'm about to get divorced). The same goes for question 4) . I chose A, "Yes", assuming it was a good movie worth seeing, but lacking the above-mentioned "context" (is it "Forrest Gump" or "Delta Force IV"?) any answer you give it's just to satisfy the interviewer. And yes, people DO go and see movies that they know they're bad. A little less reading on "behavioral economics" or "animal spirits" and a little more reading of Hume will do everyone a ton of good.

nl 15y ago
The questions aren't science - that statistics that predict that most people will get them wrong are. Or rather - the theories which predict how people will (statistically) make sub-optimal decisions in certain circumstances are science: they make predictions that can be tested.
- Radim 15y ago
  I believe paganel is referring exactly to the "sub-optimal decision" part, in the explanation (not the questions themselves). And yes, this quiz seems a load of underspecified cargo-cult horsecrap. I surely hope the original research was of higher quality.
  - nl 15y ago
    I believe paganel is referring exactly to the "sub-optimal decision" part, in the explanation (not the questions themselves). Can you expand on that? The point of the article is that people will (statistically) get the questions wrong, or at least have biases in their responses. For example, in the first question the grandparent poster may well have had good reasons to answer as they did, but that doesn't discount the fact that most people are vulnerable to the attribute substitution bias[1]. And yes, this quiz seems a load of underspecified cargo-cult horsecrap. That's to be expected isn't it? They are trying to show five different statistical effects in five questions. To do it properly they would have to ask 10's of questions for each one, but I don't think that would work for a Vanity Fair sidebar article. Instead, they tried to find questions that would demonstrate the principles to most people. [1] http://en.wikipedia.org/wiki/Attribute_substitution http://en.wikipedia.org/wiki/Attribute_substitution

gizzlon 15y ago

The ticket questions is dumb. Has anyone heard a better version of it? At least to me, the important factors are: Do I remember where I'm suppose to sit? Would It be immoral to attend even if I lost the physical stub?

srl 15y ago

After the first question: after being primed not to trust my instincts, I decided (SPOILER) it couldn't be (a), and thus guessed (c). Which I suppose reflects another important cognitive bias.

berdario 15y ago

What does "How many dates did you have last month?" even means? Maybe I've a different notion of "date" from the rest of the people... but I'd assume that if you've a stable relationship with a significant other, then your number could be definitely higher than 5 in a month... otherwise, if it's considering all the different dates with all potential partners, 3 to 5 seems an number exceptionally high (in fact: I've got a grand total of 0 dates over the last 24 months, and I'm quite sad about that)

kstenerud 15y ago

Their interpretation of question 5 is just plain wrong. Aside from the fact that losses DO hurt more than gains (because you need to alter your plans to accomodate losses), the assumption of equivalence is fundamentally flawed. The problem is that the two halves are inverted. Correcting: 5a: Your base position is $900, and you have the option of risking $900 for a possible gain of $100. You have a 90% chance of gaining $100 (a trivial amount) and a 10% chance of losing $900 (a big amount). 5b: Your base position is -$900, and you have the option of risking $100 for a possible gain of $900. You have a 90% chance of losing $100 (a trivial amount) and a 10% chance of gaining $900 (a big amount). Since you're an individual, and not a statistical average, there's no middle ground in the values gained or lost. So the rational person is right to choose A, then B. The two are NOT equivalent to an individual. The trivial amount makes little difference as a gain or a loss, but the big amount does.