21 ms·

A Quick Puzzle to Test Your Problem Solving

• rtkwe 11y ago
  Wonder how many people here immediately knew what the rule was? This is the standard example of to right way to test a hypothesis/theory and the power of Confirmation Bias, testing sequences that are invalid under the theory instead of testing what you think is correct.
  • rockdoe 11y ago
    I guessed that immediately (but still verified it). It was obvious it was going to be a trick question, so...
• ddlatham 11y ago
  I'm curious to see more about the distribution of questions and answers people had, and how the HN population may differ from the NYT's. There will certainly be self selection bias here, but if you're willing to share how you did with others, please enter it here: https://docs.google.com/forms/d/17e5BIL0lH8OHsGj89Zdtdl8GeCVLC8TbMxsPApGcSGg/viewform https://docs.google.com/forms/d/17e5BIL0lH8OHsGj89Zdtdl8GeCV... The result summary is visible here: https://docs.google.com/forms/d/17e5BIL0lH8OHsGj89Zdtdl8GeCVLC8TbMxsPApGcSGg/viewanalytics https://docs.google.com/forms/d/17e5BIL0lH8OHsGj89Zdtdl8GeCV... The raw answers are visible here: https://docs.google.com/spreadsheets/d/1ZxR2_eOUtNLXwgKfLO1JWhiXx-UQOxsYlKaBFe7f3m0/ https://docs.google.com/spreadsheets/d/1ZxR2_eOUtNLXwgKfLO1J...
  • fenomas 11y ago
    Cool idea, thanks! Personally, I spent most of my answers playing with the inputs. The form was happy to report that "1e1, 15, 0x10" was a valid sequence. :D
  • veli_joza 11y ago
    People familiar with unit testing and test driven development will feel at home with this kind of puzzle. That doesn't mean that they will be less biased in social/political decisions, it just means that this test will fail to prove a point.
    • ttctciyf 11y ago
      Hah :) Seriously, though, it seems a bit of leap from the existence of confirmation bias to explaining away the public outpourings of US Politicians about their financial crises and foreign policy disasters - in the absence of better data as to just why the given statements were made ascribing this to confirmation bias seems itself open to accusations of confirmation bias! :)
    • diminishedprime 11y ago
      That's exactly what I was thinking. I (sometimes) follow TDD, and I applied it to this problem. I made sure to include negatives, 0, positives, and include primes here or there to help avoid issues with multiplication/exponentiation. After a few of these, I felt pretty confident that the rule was simple.
      • ubernostrum 11y ago
        My process was: [3 5 7] [7 5 3] [8 4 2] [5 7 3] [1 2 3] [1 1 1] [0 1 2] At that point I could've done some more to be really certain, but felt confident enough and guessed (correctly).
        • jonkiddy 11y ago
          [16 32 64] [0 0 0] [1 2 3] [3 2 1] [-1 0 1] [3 5 7] [-2 -1 0] [111 54231 9999999999999999999999] At that point, I correctly answered the question.
          • jws 11y ago
            Similar to mine, but we both actually failed. Real numbers are accepted and we did not test that.
  • maxerickson 11y ago
    I wanted to answer "Probably" to the last question about having seen a similar question before. I don't specifically recall seeing one, but it is likely that I have.
  • elevensies 11y ago
    This is a good idea. I submitted the survey but here are my tests: http://imgur.com/hYKudyl http://imgur.com/hYKudyl FWIW, I've seen this kind of game before and I was expecting it to be something simple.
  • wzdd 11y ago
    Your last question is "have you seen a test about confirmation bias before?" But the text of this test says that it's about "why no-one likes to be wrong", which means it's pretty obviously about confirmation bias (and therefore that the test-taker should be wary about just confirming their first intuition).
  • pitchups 11y ago
    Intersting that the split of correct/wrong answers from the HN crowd is 78%/22% - the exact opposite of the general population : 22% / 78%! The HN community does think different :)
    • snowwrestler 11y ago
      Or at least, the portion of the HN community who reported their results, reported that they think different. :-)
      • JFoss117 11y ago
        Yep, I'd guess this is a massive bias. The people who get it right are much more apt to report and show how smart they are
    • philh 11y ago
      If you're taking that from the article, I think you misread it. > Remarkably, 78 percent of people who have played this game so far have guessed the answer without first hearing a single no. Some of those 78% probably got it right, and some of the remainder would have got it wrong.
      • jessaustin 11y ago
        Some of those 78% probably got it right... In what sense? How did they know that e.g. (1, 1, 1) didn't work, unless they tried it?
        • philh 11y ago
          In the sense that they guessed the rule correctly. The correct answer was the dominant hypothesis in my mind before I tried any sequences, and others here report the same.
    • teahat 11y ago
      The article doesn't actually specify that 78% guessed correctly or incorrectly, just that 78% guessed without ever entering an invalid sequence.
  • nzealand 11y ago
    This crowd needs one more question: How many "I know the answer, but can I find a flaw in their code" questions did you ask?
  • jerf 11y ago
    This is now the fourth time I've seen this exact rule. Another example: https://www.youtube.com/watch?v=vKA4w2O61Xo https://www.youtube.com/watch?v=vKA4w2O61Xo It's probably not "fair" to say I got it in zero... but I did. :) Now that the NYT has done it, this puzzle has probably attained enough popularity now that you really ought to change it up a bit now if you're going to run it yourself. Granted, the space of hypotheses as simple as "increasing/decreasing" is pretty small, but your ability to fool people with the first sample run is almost unbounded, so that helps.
    • Verdex 11y ago
      Similar story here. I got it in zero because this problem shows up at the early part of HPMOR. I suspect that the basic idea behind it is about right (people who insist on failures before committing to a theory will probably do "better"). But it seems to me that this test will be best at selecting people who've seen it before and can pretend they didn't (or even remember to ask negative questions when someone asks you to guess three numbers to get the job).
      • MichaelDickens 11y ago
        Yeah, I'd like to see other sorts of questions that you can only get right by looking for disconfirming evidence, but that don't have anything to do with choosing a sequence of numbers.
  • stillsut 11y ago
    to be pedantic: "monotonically increasing" is incorrect. It should be "strictly increasing"
  • deleted 11y ago
    [deleted]
  • hammock 11y ago
    Guessed after just one "No" 3 9 27 yes (is it exponential series?) 4 16 64 yes (is it only odd numbers?) 5 7 9 yes (is it any numbers of the same parity?) 6 7 8 yes (is it any set of increasing numbers?) 6 7 6 no (just to confirm that it's x<y<z, and not something like x<=y<=z)
    • Falcon9 11y ago
      -2 -4 -8 (is it increasing distance from zero?)
      • hammock 11y ago
        You're right, I made the assumption that our domain was natural numbers- which seems to be an uncommon one so far among the HN crowd
    • Retric 11y ago
      6,7,7 or 1,1,1 satifies x<=y<=z not 6, 7, 6
  • Jugurtha 11y ago
    My answer was: "The sequence is of an increasing real variable, where each subsequent value is greater than the preceding value. It's monotonically varying." When I clicked "I think I know it", nothing happened. I don't want to click their "I don't want to play; just tell me the answer". But it seems like the right answer. I can't answer your form question if it is the right anwer, since I haven't clicked on their link and don't know for a fact whether it is or not. Although I used the wrong term, it's strictly increasing.
  • abandonliberty 11y ago
    There's a selection bias - those of us who got it right are more likely to fill that out (: As a result we can't really rely on overall accuracy, but we can break it out by yes/no to account for the selection bias to get a profile for how a HN correct and incorrect differ.
  • larkspur 11y ago
    If you look at the raw answers, a lot of people who guessed the wrong answer according to: "What answer did you give?" said they got it right: Right? What answer did you give? Yes, double the previous number Yes, Each number much larger than the previous Yes, sequence must always increase by 1 Yes, Powers of 2 Yes, h = 2n, i = 2(n+1), j = 2(n+2) . n is an integer Yes, Powers of 2
  • Myrmornis 11y ago
    It would be relevant to include a histogram of #yes_answers - #no_answers in the summary, to test whether people are biased towards positive rather than negative tests. I think the raw data suggests that it does although I totally failed to create a histogram in google spreadsheets within 5 minutes.
  • beatbrokedown 11y ago
    My fuzzing went: 8 4 2 (no) 1 2 3 (yes) 1 1 1 (no) 1 100 123 (yes) 1.0 1.1 1.2 (yes) answer: incrementing numbers
  • scuba_man_spiff 11y ago
    [ 456 , 42 , 48 , No ] [ 1 , 2 , 3 , Yes ] [ 3 , 2 , 1 , No ] [ 45 , 46 , 47 , Yes ] [ -1 , 999 ,1238978 , Yes ] [ 1+1 , 1+2 , 1+3 , Fail ] [ 7 , 654 , 653 , No ] [ 7 , 654 , 655 , Yes ] [ 1.1 , 1.2 , 1.3 , Yes ] [ 1 , e , pi , Fail ] [ 1 , e , 42 , Fail ] [ -1.1 , -1.0 , -0.9 , Yes ] [ 2i , 3i , 4i , Fail ] [ 1 , e , pi , Fail ] [ 0 , 1 , 2 , Yes ] [ -1 , 0 , 1 , Yes ] If anyone's tallying
  • scuba_man_spiff 11y ago
    [ 456 , 42 , 48 , No ] [ 1 , 2 , 3 , Yes ] [ 3 , 2 , 1 , No ] [ 45 , 46 , 47 , Yes ] [ -1 , 999 ,1238978 , Yes ] [ 1+1 , 1+2 , 1+3 , Fail ] [ 7 , 654 , 653 , No ] [ 7 , 654 , 655 , Yes ] [ 1.1 , 1.2 , 1.3 , Yes ] [ 1 , e , pi , Fail ] [ 1 , e , 42 , Fail ] [ -1.1 , -1.0 , -0.9 , Yes ] [ 2i , 3i , 4i , Fail ] [ 1 , e , pi , Fail ] [ 0 , 1 , 2 , Yes ] [ -1 , 0 , 1 , Yes ] If anyone's tallying
  • eru 11y ago
    Thanks for setting up the survey! Could you perhaps move to a bar chart instead of pie charts?
    • ddlatham 11y ago
      Don't think that's an option for Google docs - but feel free to take the data and share one.
• m4r71n 11y ago
  Derek Muller of Veritasium has done this test over a year ago on people in one of his videos: https://www.youtube.com/watch?v=vKA4w2O61Xo https://www.youtube.com/watch?v=vKA4w2O61Xo I've given the test to various people since then and never once came across someone who'd guess it right away, or within a short period of time. The breaking time came usually after a few minutes when they gave up and started throwing out random numbers that coincidentally did not meet the rule. Once you hit the first "No", it took a very short time to figure out the rule for almost everyone.
• deleted 11y ago
  [deleted]
• bediger4000 11y ago
  The official answer to this puzzle makes a huge assumption: that there is one correct answer. There is not one correct answer. (x, 2x, 4x) gives you a "yes" every time, therefore it is a correct answer, at least as automatically checkable, and there's an infinite number of such tuples. To find a "no" you're reduced to random guessing. That's not a puzzle, that's crap. The confirmation bias material might be true, but the puzzle does not illustrate it.
  • ddlatham 11y ago
    The puzzle is not "Find a rule that matches everything you tested". It's figure out what rule they are using. It may be true that there are multiple ways to state the rule, such as "x_2 >= x_1 + 1 and x_3 is >= x_1 + 2" but they are equivalent. Try the sequence: 0,0,0. It would give "yes" for (x,2x,4x), but the actual rule gives it "no".
    • NLips 11y ago
      Or make x negative - the rule gives it "no".
  • upquark 11y ago
    Your task is to learn a class of objects by example. Without seeing any negative examples, you're unlikely to learn the right class (which is unique, it is precisely the class of increasing sequences of length 3). (x, 2x, 4x) is not "a correct answer", as it does not describe the class you were supposed to learn to distinguish.
  • jpollock 11y ago
    Well, yes, in this situation random guessing would be very likely to provide an immediate check on the solution. (1x, 2x, 4x), as indicated in the video below, is not sufficient. It represents a subset of the values that are valid. Think of it this way. When asked to write a unit test, do you only test the positive outcomes? No, you test to make sure the failures are as you expect as well. Otherwise, you are likely to have what you think is a failure end up as a success.
  • 11y ago
• tmd 11y ago
  It responds "No" to (10000000000000000, 10000000000000001, 10000000000000002) so the rule is not so simple after all :)
  • Kenji 11y ago
    Good old IEEE 64bit floating point numbers =) That also means that for(i=0;i<j;++i){} doesn't necessarily terminate for an arbitrary j smaller than infinity, which I find hilarious.
    • mikeash 11y ago
      I'd prefer an example along the lines of: for(i = j; i < j + 2; i++) {} With your example it's easy to lose the distinction between "would eventually terminate if you had a fast computer and a lot of time" and "never terminates even in theory." Here, it definitely looks like it should always run two iterations to matter what the numbers are (as long as they're finite), but it doesn't. Either way, though, it is definitely hilarious!
      • eru 11y ago
        Thanks to undefined behaviour, C compilers can assume that you get exactly two iterations for this kind of loop on ints.
    • mark-r 11y ago
      It terminates for an arbitrary j smaller than 2^53, which is enough for most people.
  • TheSlowSmoker 11y ago
    Any idea why? Could it be an error from the input being too large? Or is there some other magic at work here... Nedit: One of the other responses nailed it. IEEE standards on 64 bit fp ops
  • oldboyFX 11y ago
    Only on hackernews :O
• deleted 11y ago
  [deleted]
  • pron 11y ago
    The purpose is not to see if people use randomized inputs, or even if they're capable of analyzing data. It is to see if people are willing to challenge their own notions. The "misleading" sample is then just right. Even in real life, no discrete categorization hypothesis should be accepted if you've only seen data matching one of the categories.
  • sp332 11y ago
    That is confirmation bias. That's the whole point.
  • rm999 11y ago
    >In order to prove their point effectively without falling in the same trap they are pointing, they should conduct the same experiment with a random example each time, not one especially created to mislead the experimentee. This is the point though - we all have preexisting beliefs, and we often don't critically question those beliefs. The [x,2x,4x] pattern injected into our minds by nytimes is playing the part of the preexisting belief. The fact that most people go forward with their theory without properly testing it is the confirmation bias. Totally randomizing the numbers wouldn't be able to make this point because three random numbers usually won't have a clear looking pattern (and hence wouldn't give us an obvious preexisting belief). > In such problems, it provides more information to test your hypothesis with data matching your hypothesis than with data not matching it. I'd argue this is often not the case. For example, in the article's Iraq example, looking for evidence that there were no WMDs would have cost very little compared to even a short war. Instead, Bush's administration searched for positive evidence anywhere they could, even from questionable sources.
    • deleted 11y ago
      [deleted]
    • deleted 11y ago
      [deleted]
• jasallen 11y ago
  There is no penalty to being told "no". That also applies to "I think I know the answer". There is no penalty for being told "no" there as well, so once you have a reasonable guess why not check it? Disagree with their analysis.
  • ddlatham 11y ago
    There is an implied penalty because it says about the answer box "Make sure you’re right; you won’t get a second chance" in contrast to "You can test as many sequences as you want" about the sequences.
    • Falcon9 11y ago
      This is absolutely correct. Perhaps Hacker News culture is such that they realize such a warning can be overcome through a refresh, or possibly an incognito browser window if a cookie is preventing a second guess. Still though, within the rules of the game, there is no drawback of testing negative sequences, but there's a clearly defined drawback of entering an incorrect answer.
• abecedarius 11y ago
  I ran this experiment for a while (code at https://github.com/darius/wason https://github.com/darius/wason, derived from http://lesswrong.com/lw/g2/positive_bias_test_c_program/ http://lesswrong.com/lw/g2/positive_bias_test_c_program/). In my logs most people seem to have gotten it right, though presumably that's because it was linked from LessWrong. For an actually-fun game like this, see https://en.wikipedia.org/wiki/Zendo_%28game%29 https://en.wikipedia.org/wiki/Zendo_%28game%29.
  • TeMPOraL 11y ago
    > though presumably that's because it was linked from LessWrong No surprise there, given that this experiment is discussed in Sequences, the link to the post present in the article you link to :).
  • harryjo 11y ago
    Thank you! I forgot the name of the game, and it is totally unsearchable.
  • deleted 11y ago
    [deleted]
  • roccaturi 11y ago
    Zendo is fantastic. Watching people unaccustomed to its sort of problem-solving struggle with and improve their methodologies for being good at the game is really enlightening.
• qznc 11y ago
  > A mere 8 percent heard at least three nos I guessed correctly with only 2 nos. Since there is no penalty for guessing incorrectly here, I felt safe enough with my theory. I might have checked for more nos, if I had to announce my theory publicly (Twitter, comment, etc). However, I also knew about Confirmation Bias beforehand.
  • rockdoe 11y ago
    I guessed correctly with only one no. You can enter all kinds of crazy random sequences which only have The Rule in common and get a yes, which seemed to be enough assurance. If you're trying to get it to say "no" but failing, is that still confirmation bias? Doesn't sound like it.
    • reagency 11y ago
      That highlights a great point: on this simple decision rule, using random test data will draw the solution more quickly than human cleverness.
  • dendriteseeker 11y ago
    I also guessed correctly with only 2 nos -- my real confirmation came from the fact that "-1, 0, 300000000" (or some number of 0s) was correct, meaning it couldn't be any really meaningful sequence.
• cabirum 11y ago
  function judgeSentence(sentence, numNo) var probablyWrong = ["doubl", "expon", "multipl", "^", "", "power", "two", "2", "twice", "as big", "nth", "rais"]; var seemsRight = ["larger", "increas", "greater", "small", "less", "big", ">", "<", "go up", "ascending"], weaselWords = ['but ', 'not ', 'odd']; Been expecting something more interesting than that
  • TeMPOraL 11y ago
    It's surprising how far you can go by such heuristics. I run an IRC bot that uses matches like this (I'm working on a proper solution right now though) to parse natural language queries and I managed to trick few people into thinking they're talking with human. As long as it's ok for 90% of most common cases, people often won't notice.
    • tomerico 11y ago
      Which is another case of confirmation bias
    • tripzilch 11y ago
      I've made a robot that screams[0]. That is, just outputs a random string of "AAAAaaa" when its name is mentioned or when somebody else screams (four or more A's). What is surprising is how basically 15 lines of python implementing these rules, invokes a very real emotional response in a lot of people :-) I named it "Wilhelm". [0] inspired by this comic http://gunshowcomic.com/513 http://gunshowcomic.com/513
  • stillsut 11y ago
    include "monoton" in probablyWrong. The sequence is not monotonically increasing.
    • thekingofspain 11y ago
      Interestingly, it would appear that monotonically increasing in general means "strictly increasing": http://mathworld.wolfram.com/MonotoneIncreasing.htm http://mathworld.wolfram.com/MonotoneIncreasing.htm It only means non-decreasing in the context of a monotonic function, where the definition, I believe, is that the derivative of the function is never <0.
• js2 11y ago
  For a more extensive version of this article, see "Mistakes Were Made (But Not by Me)": http://www.amazon.com/Mistakes-Were-Made-But-Not/dp/1491514132 http://www.amazon.com/Mistakes-Were-Made-But-Not/dp/14915141...
• Kenji 11y ago
  "We’ve chosen a rule that some sequences of three numbers obey — and some do not. Your job is to guess what the rule is." My mistake was to assume that choosing the first number uniquely defines the next ones in the sequence. Since, you know, like all the sequence puzzles I've seen before worked like that, and I didn't read it rigorously enough. Oh, by the way, the doubling thing is wrong if you use negative numbers (wrong as in it gives false positives, instead of just false negatives). But the problem definition doesn't even tell what set of numbers we're operating on. Finding the rule the sequences obey is impossible since it could be that all cases follow a simple rule except for one triplet which you're unlikely to find. It's trivially easy to fool the user into finding a wrong rule.
  • mikehawkins 11y ago
    Same here - I was also guilty of feeling a bit clever and avoiding the obvious "oh, the numbers double every time" answer. So, when I found out that if the spelling of each number was one letter longer (4 = four letters, 8 = five letters, 11 = six) I got smug, and didn't bother testing further. Good article - and a humbling experience. :)
    • TylerE 11y ago
      I went for the slightly more clever "series of powers" after 3/9/27 validated.
      • dllthomas 11y ago
        That was a hypothesis I considered, as was x,2x,4x. Invalidated both of those, and ultimately drew the intended conclusion (which is not quite "correct", given javascript numeric precision issues).
  • deleted 11y ago
    [deleted]
  • deleted 11y ago
    [deleted]
  • 11y ago
• sp332 11y ago
  I got 7 yes, 5 no, and was still wrong. Maybe I'm just dumb.
  • EvanKelly 11y ago
    I'm curious about how you came to your answer and what led you there? Were you testing a pre-supposed hypothesis that confirmed itself?
    • sp332 11y ago
      I got the part about increasing eventually, but I thought the third number also had to be the sum of the other two. I came up with the sum idea after trying (3 6 9), so only 2 tests. The idea that they had to be increasing came later. I don't have it open but I'm pretty sure one of my tests was (1 2 5) which should have tipped me off... in conclusion yes, I'm probably dumb.
      • snowwrestler 11y ago
        I know you're joking, but I think this is important: Failing the test does NOT mean a person is dumb. The point of the article is that confirmation bias seems to be a fundamental default in the way everyone thinks. Certain people with specialized training in inductive problem solving (scientists etc.) have learned to compensate. If folks think it's an issue of intelligence, then they might be willing to think "but not me, because I'm smart." (After all, many programmers believe that they are smarter than the average bear). But while programmers are well-trained to think carefully about sequences of numbers, they might be as susceptible as anyone else to confirmation bias in other areas.
        • carussell 11y ago
          I wonder how much games like Twenty Questions plays into conditioning towards this kind of approach, since the implication there tends to be that the fewer questions you ask, then the better you've done.
      • deleted 11y ago
• 725686 11y ago
  That page prints the nyt logo in the javacript console (will probably look messed up here): 0000000 000 0000000 111111111 11111111100 000 111111111 00000 111111111111111111 00000 000000 000 1111111111111111111111111100000 000 000 1111 1111111111111111100 000 000 11 0 1111111100 000 000 1 00 1 000 000 00 00 1 000 000 000 00000 1 000 00000 0000 00000000 1 00000 11111 000 00 000000 000 11111 00000 0000 000000 00000 00000 000 10000 000000 000 0000 000 00000 000000 1 000 000 000000 10000 1 0 000 000 1000000 00 1 00 000 000 1111111 1 0000 000 000 1111111100 000000 000 0000 111111111111111110000000 0000 111111111 111111111111100000 111111111 0000000 00000000 0000000 NYTimes.com: All the code that's fit to printf() We're hiring: developers.nytimes.com/careers
  • deleted 11y ago
    [deleted]
  • heartbreak 11y ago
    Every nytimes.com page displays that in the JS console.
    • stephengillie 11y ago
      Putting your engineer hiring notices under the hood is becoming a common "Easter-egg" practice. It's also a form of targeted recruitment advertising - the only people who see it are your target audience.
• progmanal 11y ago
  The attached reading material is interesting, but this question is too similar to problems where you guess the next one in the sequence and none are missing. A rule where the numbers are increasing does not explain why 3 or 5 or 6 is missing from the sequence in that version of the question that is much more common.
• danielvinson 11y ago
  I remembered this puzzle from HPMOR, but either way this is just writing a unit test.
  • Tyr42 11y ago
    Having just re-read HPMOR a few weeks ago, I could answer it right away, but there was actually a difference from HPMOR's version, which required three positive increasing numbers, while this allowed negatives.
• stygiansonic 11y ago
  Neat - as others have pointed out, I feel that being familiar with unit testing would help in this situation. Having negative test cases is just as important, if not moreso, than having "happy path" tests.
  • david_shaw 11y ago
    That's a great point. Perhaps we should show something like this to new developers who don't understand the value.
• Geee 11y ago
  Anyone tried entering letters in the boxes? I didn't. I think that would have been an unexpected twist. So, after all, I think I fall in the trap of confirmation bias that the sequence must consist of numbers only.
  • deleted 11y ago
    [deleted]
  • jazzyb 11y ago
    I entered A, B, C. The form didn't let me submit the sequence, so they are doing some verification that the entries are numbers. Rational numbers work, though: 1.1, 1.2, 1.3.
  • dEnigma 11y ago
    Well, after all they tell you that the sequence consists of numbers: >We’ve chosen a rule that some sequences of three numbers obey >Now it’s your turn. Enter a number sequence in the boxes below, and we’ll tell you whether it satisfies the rule or not.
• jjuhl 11y ago
  The same experiment: https://m.youtube.com/watch?v=vKA4w2O61Xo https://m.youtube.com/watch?v=vKA4w2O61Xo
  • zeidrich 11y ago
    What's funny is that I felt smug and self satisfied for getting the correct answer even though I watched that video when it came out.
• adamc 11y ago
  I got it, but it took a bunch of guesses before I had the pattern. It's a good example of how our preconceptions shape our answers.
• phaemon 11y ago
  I got this wrong, but oddly enough my first attempt was a No. I tried, "6, 1, 8" After that, I went with doubling and it worked 3 times with various sizes of number, so I went with that. Double bonus points if you can guess what I was testing with the first sequence (which the given numbers do satisfy).
  • o0-0o 11y ago
    Were you guessing random area codes of midwestern states? :)
    • phaemon 11y ago
      Hah! No, I'm not American :) But, in case no-one guesses, the answer is (rot13): erirefr nycunorgvpny beqre
      • reagency 11y ago
        You vastly overestimated NYT's estimate of the cleverness of their readers
        • phaemon 11y ago
          A bit cruel, but I admit I laughed at that.
• __z 11y ago
  Veritasium video on the same thing https://youtu.be/vKA4w2O61Xo https://youtu.be/vKA4w2O61Xo
• noreasonw 11y ago
  I think that the "quick" adjective in the title is purposeful misleading. You are supposed to learn quickly the most general rule, but that is not so easy because there are many possible rules that could fit such a pattern. It seems that you should be rewarded for solving the puzzle quickly and then you fall in the trap. I propose to change the title to "A puzzle to test your Generalization Abilities", and state clearly that you should try to find the most general rule that satisfies all patterns you can think of. In that case, I would expect the conclusion and results of the experiment to be completely different. So to summarize: the so "quick" adjective in the title has a very strong anchor effect. Edit: changed for grammar and to express more clearly what I think.
  • Retra 11y ago
    Maybe it should be "a puzzle many people already know the answer to" in which case the conclusion and results are already obviously biased. I was able to solve the puzzle without testing any numbers at all. Which really skews the relevance of "only nine percent of people saw three 'no's before answering."
    • crgt 11y ago
      You mean you correctly guessed without testing at all - and got lucky. I'm not sure this is the same thing as 'solving' it.
      • Retra 11y ago
        I "got lucky" in the sense that my experiences have primed me to recognize that particular kind of question. If the article had actually modified the question framing at all, rather than just copying the existing one, it maybe wouldn't have worked that way. It's the same reaction I have to the Monte Hall problem: I don't have to think or be clever to get the right answer. So naturally you couldn't effectively teach me anything by simply posing the question, and any information you collect by doing so won't accurately reflect what I learn or how well I think. You'd just be testing topical familiarity.
• _lce0 11y ago
  I thought.. F(n + 1) = n * 2
• afitnerd 11y ago
  Seems busted now - clicking the "I think I know" button does nothing. I thought the answer was: Let the first number be x. If x is 0, then the second number is 1. Otherwise, the second number is two times the absolute value of x. The third number is 2 times the value of the second number.
  • gervase 11y ago
    The answer they were looking for was: Let the numbers be x, y, and z; x < y < z must be true. Their hypothesis was that most people would guess as you did (they mentioned 78% of people did so).
  • lowmagnet 11y ago
    I had to turn off µBlock to make it work.
• phantarch 11y ago
  Funnily enough, I notice confirmation bias quite a bit in a D&D game I am currently DM of. I'm playing with a group of friends who are big into video games, and as a result they consistently seek resolutions to conflicts in D&D by way of what they know from shooters: kill everything in sight. Yes, it's at times a valid answer, but it's not the only one and it's certainly not the most interesting one. The best way that I've seen the confirmation bias dissipate from their thinking is to put them into situations where their bias just doesn't help at all.
  • mikeash 11y ago
    You must be awfully tempted to constantly give them powerful adversaries who are willing to become friends if given the smallest chance.
  • pizza 11y ago
    Maybe some positive/negative feedback built into the campaign could help; e.g. for each act of benevolence/violence, add/subtract a 'karma' point from some running total, and alter the gameplay as needed.
    • eru 11y ago
      Just don't tell them explicitly about it. (That'd be taking away all the fun.)
• Kluny 11y ago
  Decimals and negative numbers still get you a "yes" as long as you obey the rule. However, the button to submit your answer didn't work for me.
• dhruvbird 11y ago
  People try to make the fit be as tight as possible to the sample data -- the explanation is that simple. I don't buy the explanation provided in the article.
  • mpu 11y ago
    Pretty good point. That's how I felt. Additionally, this setting is probably too close to usual situations you get in school where there is little to no interaction and negative answers from the teacher are seen as failures by students. (Speaking about education in my country only.)
• binarymax 11y ago
  The puzzle is not nearly as interesting as the code being able to understand my answer! My answer: The numbers increase from left to right Application response: As you seem to have guessed, the answer was extremely basic
  • Retr0spectrum 11y ago
    I said "The numbers are in ascending order", which it didn't seem to recognise.
    • matchu 11y ago
      "ascending" is one of the words it checks for. Might've typoed it? :/
      • aylons 11y ago
        I tried, coincidentally, the exact same phrase as he did, with no effect.
  • philh 11y ago
    Yes, I'm curious what heuristics it used there. Playing the confirmation bias game on "playthroughs of the confirmation bias game": it looks like you get that message if you have a 'no' answer, and the word 'increasing' in your guess. ("Increasing by the same amount" is still accepted.) I wouldn't be surprised if other words also count. Edit: the words '<', '>', 'increas', 'big' and 'larger' seem to count. Looks like they're accepted as substrings, not just words - so 'increase' and 'increasing' are accepted. I could look for a long time, so I'm giving up now.
  • arjunnarayan 11y ago
    I simply said: "a < b < c", with no other words, and it gave me the same answer as yours.
• graycat 11y ago
  Typical NYT psycho-babble: I got the right answer but with several Yes answers and also several No answers and without their confirmation bias and other nonsense. And, I do know about "problem solving", thank you, with some of my solutions responsible for my peer-reviewed published papers and Ph.D. in some STEM work at a world class research university. Sorry, NYT: I don't regard your organization as able to "test" "problem solving". Also "the rule" likely makes no sense since it is not clear that there can't be multiple rules that describe their black box. Indeed, it has long been known that there are some fundamental challenges saying what is in a black box just from looking at pairs of inputs and outputs -- of course, can't expect the NYT psycho-babble click bait writers/editors to know that. Once again the NYT looks like an organization down for a count of 8 or 9 out of 10. NYT: Give up the social pseudo-science psycho-babble and other nonsense nearly as bad as leech bleeding, move out of the 18th century and at least into the 20th century, e.g., with its standards of information safety and efficacy illustrated by mathematics, mathematical physics, chemistry, biology (e.g., DNA), engineering, medical science. Come on, NYT: Drop the 18th century leech bleeding information safety and efficacy. Have you no shame to look so foolish in front of millions of people?
  • snowwrestler 11y ago
    Jeez man, it's just an article.
    • graycat 11y ago
      It's a grotesque insult to you, me, and all of HN with NYT deliberately exploitative psycho-babble where the NYT believes that the exalted, highly self-esteemed NYT is way above their audience that can be so exploited with such nonsense. It's an insult, a deliberate arrogant insult. Haw haw, hee, hee, the NYT, the highly superior NYT, highly exalted, is going to insult their audience, milk it like their own herd of cows, pass out smelly bait for an ad hook, pat themselves on their backs as they erect themselves as authoritative arbiters of what it means to be able to solve a problem -- sewage. You are being insulted as much as if you were just walking down a sidewalk and the NYT saw you coming and poured sewage on you from two floors above. Put up with that treatment if you wish, but know that you are being insultingly exploited by the psycho-babble version of sewage.
• deleted 11y ago
  [deleted]
• TerryADavis 11y ago
  http://www.templeos.org/Wb/Doc/CharOverview.html http://www.templeos.org/Wb/Doc/CharOverview.html
• dude_abides 11y ago
  I'm a data scientist, and it relieved me no end that I got this one right: http://i.imgur.com/V5oJ4i4.png http://i.imgur.com/V5oJ4i4.png I would have had second thoughts about my career choice if I got this wrong :) The correct approach for any data modeling problem is to think in terms of entropy. Each subsequent approach should minimize entropy, until you reach diminishing returns.
  • taigeair 11y ago
    I got so many nos. I can't believe that "Remarkably, 77 percent of people who have played this game so far have guessed the answer without first hearing a single no." That's crazy.
  • Sadgrinner 11y ago
    Isn't your answer technically wrong, though? The sequence is not monotonically increasing. It's strictly increasing. If you test [1, 1, 2] or [1, 1, 1] or [1, 2, 2], you'll get "No" answers even though those sequences are monotonically increasing.
    • thaumasiotes 11y ago
      Not so. http://mathworld.wolfram.com/MonotoneIncreasing.html http://mathworld.wolfram.com/MonotoneIncreasing.html
      • ddlatham 11y ago
        So. http://mathworld.wolfram.com/MonotonicFunction.html http://mathworld.wolfram.com/MonotonicFunction.html https://en.wikipedia.org/wiki/Monotonic_function https://en.wikipedia.org/wiki/Monotonic_function
        • thaumasiotes 11y ago
          You think the definition of "monotonic function" is more relevant to the meaning of "monotonically increasing sequence" than the definition of "monotone increasing" is? [1,1,2], [1,1,1], and [1,2,2] are not monotone increasing. They're also not monotonic functions.
• pmelendez 11y ago
  I failed it again... Every time I take a test like this I ended being fooled by the confirmation bias.
• harryjo 11y ago
  Veritasium covered this classic puzzle / bias test nicely, last year. https://www.youtube.com/watch?v=vKA4w2O61Xo https://www.youtube.com/watch?v=vKA4w2O61Xo
• ammaar 11y ago
  for anyone who hasn't seen it Veritasium did a video on it. https://www.youtube.com/watch?v=vKA4w2O61Xo https://www.youtube.com/watch?v=vKA4w2O61Xo
• mastre_ 11y ago
  -10, 0, 0.01
• sambe 11y ago
  The "Check" buttons weren't enough of a hint for me and I jumped into "This is a numerical reasoning problem" mode. I'd argue that this kind of situational bias is as much a factor here as confirmation bias.
• yequalsx 11y ago
  Math person here. I'm curious to know if anyone used decimal numbers in their tests and if negative numbers were used. The rule is increasing real numbers and one can guess that the rule is increasing numbers without realizing this includes all real numbers and not just integers. In addition to getting it right did you use an exhaustive set of tests?
  • MalcolmPF 11y ago
    I tried a negative series and a decimal series, just in case the rule was increasing natural numbers. I tried very large numbers to see if there was a limit to the rule, and a series that had large contrast in between each element. For fun I tried to see if the app would recognize "pi", "i", or "e", but, perhaps unsurprisingly, it did not.
    • yequalsx 11y ago
      I would have checked complex numbers for fun but there is no reasonable ordering of the complex numbers in the way there is for real numbers. It's too bad it didn't recognize pi or e.
  • Infernal 11y ago
    I did test with negatives, but I did not think to test with decimals. 15 yes's, 6 no's, and I successfully determined the rule.
  • pbnjay 11y ago
    I got it right, using decimals - I was probably a little too exhaustive... http://imgur.com/6adtvqz http://imgur.com/6adtvqz
  • perlgeek 11y ago
    Yes and yes. It wouldn't activate the "check" button with complex numbers.
    • yequalsx 11y ago
      There isn't a way to check order for complex numbers in any reasonable way.
• ljk 11y ago
  did anyone else only try even numbers? guess "increasing even numbers" pretty interesting how in-the-box my thought process was
• deleted 11y ago
  [deleted]
• FilterJoe 11y ago
  While the HN crowd mostly gets this right when framed as a math puzzle, my guess is that confirmation bias is alive and well in high tech just like in any other field. One example: Young 20s entrepreneur vs. early 50s entrepreneur. Without knowing anything about either person, which startup is more likely to succeed? Even if you have the business plans for both, and you meet both - which one are you going to be more skeptical about as you evaluate which one of them gets funding?
  • eru 11y ago
    I'd trust the older guy.
• damoncali 11y ago
  Cool. The funny thing is I inserted a constraint of my own invention without even realizing it: "Use the fewest number of examples possible." Of course, this meant failing miserably, and was nowhere in the problem statement. Perhaps that's an additional factor - not exactly confirmation bias, but not unrelated.
  • Nimitz14 11y ago
    Eliminating assumptions baby. https://www.youtube.com/watch?v=9-9VLVkm8R4 https://www.youtube.com/watch?v=9-9VLVkm8R4
  • aroman 11y ago
    this is exactly what I did. I wonder what the psychology/cognitive science explanation of this might be.
• aaronbrethorst 11y ago
  More or less related: when I'm looking for constructive criticism from someone, I'll ask them "what do you dislike about this?" or "what's wrong with this?" instead of "what do you think?" I tend to get much more interesting and useful feedback this way.
• gesman 11y ago
  After total of 13 answers including 6 "no", I guessed it right :)
• pbnjay 11y ago
  If anyone's curious about the raw numbers, I found the counts here: http://int.nyt.com/newsgraphics/2015/2015-06-26-rule-guessing-game/numno.json http://int.nyt.com/newsgraphics/2015/2015-06-26-rule-guessin... {"count": 27723, "numNo": 0, "share": 0.7716051100782098} {"count": 2921, "numNo": 1, "share": 0.08129922903504133} {"count": 1883, "numNo": 2, "share": 0.05240891758746417} {"count": 1285, "numNo": 3, "share": 0.035764980934621056} {"count": 880, "numNo": 4, "share": 0.024492749589468118} {"count": 525, "numNo": 5, "share": 0.014612151743716774} {"count": 288, "numNo": 6, "share": 0.008015808956553202} {"count": 156, "numNo": 7, "share": 0.004341896518132985} {"count": 108, "numNo": 8, "share": 0.0030059283587074506} {"count": 56, "numNo": 9, "share": 0.0015586295193297892} {"count": 45, "numNo": 10, "share": 0.001252470149461438} {"count": 11, "numNo": 11, "share": 0.00030615936986835146} {"count": 22, "numNo": 12, "share": 0.0006123187397367029} {"count": 8, "numNo": 13, "share": 0.00022266135990425562} {"count": 4, "numNo": 14, "share": 0.00011133067995212781} {"count": 5, "numNo": 15, "share": 0.00013916334994015976} {"count": 3, "numNo": 17, "share": 0.00008349800996409585} {"count": 1, "numNo": 18, "share": 0.000027832669988031953} {"count": 3, "numNo": 20, "share": 0.00008349800996409585} {"count": 1, "numNo": 21, "share": 0.000027832669988031953} {"count": 2, "numNo": null, "share": 0.000055665339976063906}
  • buckbova 11y ago
    Interesting totals. I had 4 No's and would have had at least 5 had I considered negative digits.
• nsxwolf 11y ago
  So, am I supposed to feel bad if I assumed it was some tricky, hard to figure out function? It just reminded me of those questions on the ACT or whatever and I froze up and got frustrated. Am I a dim bulb?
• nothrabannosir 11y ago
  My mathematical logic is rusty, but if I recall correctly, Gödel's incompleteness theorem basically states that it is impossible to solve this kind of question. No matter how many tests you run, there will always be an uncertainty. An incredibly stupid example is that the rule could be "yes for strictly increasing, OR if one of the numbers is -18273192783127897981." You'll never know. I understand this is contrived, especially when the test subject doesn't know. But if you do realize this while doing it, it makes the test a little frustrating.. EDIT: I see people are making a connection with unit testing, and the irony is poetic. This is precisely the problem Dijkstra was talking about when he said that "Testing shows the presence, not the absence of bugs."
  • the_af 11y ago
    It's not ironic. Dijkstra's assertion is, in a convoluted way, related to the main point of the quiz and the article :)
  • monjaro 11y ago
    This has nothing to do with Gödel's incompleteness theorem. It's much simpler than that: https://en.wikipedia.org/wiki/Wittgenstein_on_Rules_and_Private_Language#The_rule-following_paradox https://en.wikipedia.org/wiki/Wittgenstein_on_Rules_and_Priv...
    • GhotiFish 11y ago
      Thanks for the link! That was an interesting read, but I don't think I understand the premise of the argument. From the article ... It is perfectly consistent with your previous use of 'plus' that you actually meant it to mean the 'quus' function, ... That may be true, but only if you assume I'm not referring to the plus derived from the axioms of principa mathematica. I am. Is it then the question that when I refer to the principa mathematica that I'm actually referring to principa quus? If I then describe axiom 1 can you not know my words are referring to axiom quus? It seems the only power of this assertion is that language provides no absolute common ground. Am I understanding this correctly?
• shmageggy 11y ago
  This phenomenon has had a profound effect on the history and philosophy of science. There have been entire schools of thought based on verification of hypotheses, and entire movements based on refuting those schools. The most effective strategy in this puzzle(and the one that is unintuitive for many) is to systematically generate alternative hypotheses and falsify them. Karl Popper claimed that this method is actually at the core of how we gain scientific knowledge, and his brand of philosophy of science is the most popular and arguably the most successful today.
• akshay_cornell 11y ago
  The NYT test is wrong. I have found a case where their "rule" is violated. By using following three numbers I got No when the result should have been YES, just goes on to show that there are too many "YES" Men. I think the reasoning and logic employed in this article is shoddy. The reality is much more complex and we do not have access to a perfect oracle. 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003
• harryh 11y ago
  Reminds me of the game mastermind which I loved playing as a kid. Here's a javscript version: http://www.archimedes-lab.org/mastermind.html http://www.archimedes-lab.org/mastermind.html
• moo 11y ago
  I've heard of intelligence failures to explain the Iraq war, now this author says it was confirmation bias which caused a completely erroneous justification for the invasion of Iraq. I think the author has confirmation bias in too easily using the term to explain government and corporate policy choices which have been based on false justifications.
• pbreit 11y ago
  I guess I'm the opposite: I got more No's than Yes's. My immediate sense was to find No's. I guess that's why I like 1 star reviews.
• drhdylan88 11y ago
  Veritasium, a pretty interesting youtube channel, posted a video on this experiment a while back. I found the discussion afterwards to be more thought-provoking than this article. https://www.youtube.com/watch?v=vKA4w2O61Xo https://www.youtube.com/watch?v=vKA4w2O61Xo
• theVirginian 11y ago
  I don't seem to be able to submit my answer or follow the link to see the answer for "just tell me the answer" is this supposed to be a trick or is my browser just not capable of properly following those links? Using Chromebook.
• sushirain 11y ago
  My explanation to the scarcity of "No"s: people are used to seeing in these puzzles mostly sub-types of increasing series: exponential, linear, etc. By the time they had ruled out these sub-types, and had resorted to guessing "ascending", they wouldn't have encountered even a single No. I predict that if the rule was narrower, like "exponential", much more guesses would have yielded No's.
• bluker 11y ago
  My first assumption was x,2x,2(2x) - then I tested 0,0,0 and got a NO which disproved my first assumption... the answer is obvious after. Did anyone else test 0,0,0?
• 0xdeadbeefbabe 11y ago
  Confirmation bias affects Corporate America and Government Policy, but not science? Talk about confirmation bias :)
• flint 11y ago
  Yup - Dick Cheney!
• Gabriel_Martin 11y ago
  My process was [1, 2, 3], and then I guessed each number is greater than the last. It was totally a possibility that they wouldn't apply the simplest rule, but I felt it highly unlikely. This "rule" is a meme of the rationality community, especially given the example, so it seemed pretty likely that it was sequential numbers.
• jessaustin 11y ago
  Have large numbers of HN people really not seen this riddle before? Maybe I read weird things.
• Lorenzo45 11y ago
  Wish I could have done this with no previous knowledge of the puzzle, I knew it right away because I saw the exact same problem in a Veritasium video.
• ernsheong 11y ago
  We should all be doing good on this one if we have been practicing making our test cases are red first before turning them green :D
  • lugg 11y ago
    Looking at my result there, seems to be something I need to start doing. TIL :P To be fair, I did check 3/6/12 - just to make sure it was double and not powers of two. Guess that's the articles point though isn't it!
• mc808 11y ago
  This looked like a puzzle I had seen before, so I assumed this was the case (testing a few sequences just to verify) and turned out right. I guess the conclusion is that if a problem looks suspiciously like one you've encountered before, there's a good chance that they are the same or similar. The world is self-organizing, not completely random where you must obsessively second-guess your accumulated wisdom.
• brianwillis 11y ago
  This doesn't seem to work right for negative numbers. The article says the rule is "each number must be larger than the one before it", but if you try -2 -4 -6 it says that pattern doesn't match the rule. Maybe I'm just being pedantic here, but last I checked -6 was larger than -4.
  • StevenXC 11y ago
    "Larger" in this context means "greater than" in the usual ordering of the real numbers.
  • Tloewald 11y ago
    It works for floating point numbers -- 0.01 0.02 0.04 for example. So it's a geometric series that has to start with a positive number and doubles. (The submit button and the show the answer button were broken for me.)
    • eru 11y ago
      That's unintentionally hilarious.
      • Tloewald 11y ago
        Yeah I was wrong in exactly the way the article expected :-) But the submit button being broken did kind of suck.
• deleted 11y ago
  [deleted]
• elektromekatron 11y ago
  I liked this puzzle. It got me. I am thankful for it.
• elwell 11y ago
  After programming daily for more than a decade, I question my assumptions annoyingly often.
• yellowapple 11y ago
  I think the first-known matching pattern plays a huge role. The original 2,4,8 sequence, for example, locked me in immediately to doubles of the previous number (causing me to test 1,2,4 and 7,14,28 and such). Had it been a different starting sequence (like 3,9,27), I might've based my guesses differently. Same for 1,2,3. In other words, first impressions really are important.
• arikrak 11y ago
  the questions is somewhat ambiguous. it would be more interesting to see what happens of people really understood the question and if they had a real motivation to get it correct.
• smilefreak 11y ago
  Does laziness have anything to do with the responses? You get a rule that seems to work and so you seek the reward early. It takes effort to prove yourself wrong. I was trapped by this and guessed it was exponential series n^1,n^2 etc for n starting at greater than 2. While technically true this was not the rule they had in mind.
  • eru 11y ago
    Why is it technically true?
    • smilefreak 11y ago
      As in every number in that set is a subset of the larger set of x < y < z. Poor language choice it's not true, yes I was wrong. I am just curious as to how much laziness and not necessarily confirmation bias has to do with the result. If getting it wrong had some kind of penalty or getting it right had some kind of reward ( money etc. ..), how much better would people do then?
      • eru 11y ago
        Yes, it seems most people's rules actually were subsets of the true rule. Laziness is involved to a certain degree. Though I don't know how much---no one is forcing anyone to do the puzzle, so the truly lazy filter out, too.
• MarkMc 11y ago
  The comments here suggest people are missing the full significance of this problem. It's not just a cute number puzzle - it demonstrates a profound human weakness that has a deep impact in everything we do. 1. People that think having a gun in the house makes it safer will not try to design an experiment designed to demonstrate the opposite. 2. People who think organic food is better for you than regular food will not try to look for evidence that the two types of foods are equally healthy. 3. An Israeli who believes the area where he lives was uninhabited before 1948 is not going to think about what kind of evidence would contradict that belief. I'm not saying the views above are incorrect. It's just that we are all guilty of falling in love with our beliefs when they should be mere acquaintances. Hence the quote, "People don't change their minds. They die, and are replaced by people with different opinions." [1] [1] http://www.paulgraham.com/quo.html http://www.paulgraham.com/quo.html
• PSeitz 11y ago
  Nice game, got it right after 7 yes and 7 no
• mordrax 11y ago
  They don’t want to hear the answer “no.” In fact, it may not occur to them to ask a question that may yield a no. So the author's obviously never heard of sanity checking, in fact that's the second thing that I always do once I confirm a solution is to confirm it's not a fallacy. Having said that, my solutions were -10 -20 -40 -10 -8 -4 1024 1026 1030 and I said it was +2, +4 and got it wrong!
• cristianpascu 11y ago
  I understand the power of confirmation bias. I believe it to be natural for anyone. It's perfectly normal to seek an explanation that fits the already built cognitive structures, developed through experience. It's unreasonable to simply jump into new paradigms everytime we encounter a new fact. It takes time to prove that it doesn't fit, and then we start looking for new explanations. However, the test simply required a possible solution. There are plenty solutions and it's absurd to think they have the simplest one. The simplicity of the rule is subjective, in that is evaluated differently by different people. The famous 'simple but no simpler' is relevant here. As long as we were not told to look for the simplest solution, ALL solutions are equally probable. That being the case, I started to with the first solution that popped into my mind. I sticked with that because of my psychological state. Some searched for other solutions. I don't think that getting an YES was the main driving force. Of course it feels good to get an yes. This is fundamental in human relations. But it's not the whole story. People do not disbelief globar warming because they want to get an YES. The reason is much deeper. Just as many, so many people go to the wave of climate change because it's fashionable, it makes them feel good, accepted , part of the mainstream. Being a climate change denier is being a disident this day (not my flavor of disidentism), and being a disident is not for everyone. And perhaps disidents picking their fight have complicated reasons for doing so. I went with the doubles.
• vincston 11y ago
  Sadly ive already seen Veritasiums test. so i was biased.
• nerdo 11y ago
  Reminiscent of the folding table libertarians with their questionnaire and that political cartesian coordinate chart. "You answered that 'it's wrong to steal', [...psychobabble...], on this science graph it appears you've always secretly been a libertarian, we meet at the cinnabon on sundays".
• decisiveness 11y ago
  Constraints allow for only integers <= 9999999999999999 and >= -9999999999999999 which is interesting considering (-)10000000000000000 through (-)9223372036854775807 are also within the bounds of a 64 bit integer. It's ironic that these facts are not mentioned considering the article is about confirmation bias.
• Gravityloss 11y ago
  Anyone played Monkey Island 2? There's a puzzle with a doorman, there's a few distracting clues where the answer is actually very simple. It doesn't offer confirmation bias though, and it took some time to figure it out. I consider it a very similar test. So one could actually construct such a test without the confirmation bias part, and then look at how long it takes for people to realize the simple model.
• limeyx 11y ago
  Got it right with 7 nos.
• SZJX 11y ago
  Umm... Not sure what's so special about it. Isn't that what all programmers and science people do all day?