41 ms·

Dark Motives and Elective Use of Brainteaser Interview Questions

290 points by 33degrees 8y ago

spython 8y ago
This reminds me of Jewish Problems[0], designed to "prevent Jews and other undesirables from getting a passing grade" at the entrance exams to the math department of Moscow State University. [0] https://arxiv.org/abs/1110.1556 https://arxiv.org/abs/1110.1556
- golergka 8y ago
  Wow, I never knew they actually published a paper on it. Coming from a Moscow math school that traditionally had a lot of jewish students, most of whom went on to MSU to study math, I remember studying these exact problems - although in my time, this filter was supposed to long been gone.
- hogwild 8y ago
  For a second there I thought you were talking about the interview process for becoming a Jew when they weed out undesirables.
- markatkinson 8y ago
  This was discussed in Edward Frenkels book Love and Math
- taurath 8y ago
  As a self-taught programmer that has had to go back to get "cs fundamentals", half of interview questions feel like that - if you just know about x or y data structure for this particular edge case, everything becomes simple/easy.
diN0bot 8y ago
interesting that “teasers” are viewed as callous and abusive and yet analytical problem solving is viewed as predictive. fine line.
- frandroid 8y ago
  A brain teaser usually has nothing to do with the work... What would asking about the shape of manhole covers have to do with OOP?
  - darawk 8y ago
    The theory would be that it allows the interviewee to demonstrate abstract reasoning skills, which are useful in many domains.
    - prolepunk 8y ago
      Also the interviewee needs to have exactly the same assumptions about the world as interviewer (or whoever come up with the brainteaser) and the knowledge of the same basic facts. I mean how do you even approach questions like -- how many snowflakes on the south pole? how many fish in all of the oceans? how much grass is in Mongolia?
      - darawk 8y ago
        I think that's the point of the question: to see how you approach it. The 'correct' approach that they'd be looking for is 'Fermi estimation': https://en.wikipedia.org/wiki/Fermi_problem https://en.wikipedia.org/wiki/Fermi_problem
        ubernostrum 8y ago
        No, the point is to try to rationalize cargo-culting. Fermi did this, and Fermi was brilliant, so if we require candidates to do this we'll end up hiring people who are as brilliant as Fermi! Here's a brainteaser: spot all the flaws in that chain of "reasoning".
        darawk 8y ago
        Um, no. It's not cargo-culting. It's the correct way of estimating things. It has nothing to do with Fermi having done it. Some things actually have correct answers. 2 + 2 = 4, is not cargo culting.
rossdavidh 8y ago
So, there's a hidden value to these sorts of questions, but it is useful to the interviewee, not the interviewer. If they use lots of these questions, you probably don't want to work there, or at least not for the person who's asking those questions.
- welcome_dragon 8y ago
  I disagree, as long as getting the right answer isn't important. I flat out tell people I don't care if they get the right answer, that I just want to hear their thought process. Such questions used in this way are extremely valuable.
  - athenot 8y ago
    I do the same thing. It's how the thought process works that's interesting, not the result. There is no problem or teaser whose answer is both short enough to be devised during an interview AND is representative of how a candidate would perform in real projects. Our field deals with complex systems in many dimensions. Instead, looking for how a candidate approaches a problem, do they just go for the obvious solutions, how creative will they go, any experience they can draw on, how well they overcome unknowns, etc.
    - gaius 8y ago
      It's how the thought process works that's interesting, not the result. I wonder how many interviewers have the necessary background in clinical psychiatry to analyze a candidate's thought process, or if that's just more handwaving away hazing.
  - nomel 8y ago
    I think a good compromise is to modify the questions so it's actually applicable to the context of the job.
  - sidlls 8y ago
    They're valuable for what? Showing off a thought process in a high pressure one hour meeting with a judge watching you?
rrggrr 8y ago
The single most effective interview question I ask: "What do you want to do?". I drill down on that to a granulator level as I try to make sure the tasks, duties and overall position matches what the candidate really wants for themselves. Its followed-up with a "How would you do it?", again persisting in the discussion (not interrogation) until we both are comfortable with the sensibility and symmetry of the answers.
- wonderbear 8y ago
  I'm going to try this. The questions my team comes up with haven't been very satisfying and this seems like a good way to find out what they do know vs what they don't.
  - rrggrr 8y ago
    I learned this from one of the best recruiters in finance. I can't take credit for it.
- pinewurst 8y ago
  That may be a good conversational start but I'm less certain that anyone really knows at any deep granularity what they want to do (professionally). There are a lot of roads to autonomy, mastery and purpose, and a lot of speed traps/bad exits. My current job, which I've just resigned because it became so awful, was on the surface what I wanted to do, until it changed. It would be difficult to discuss the before and after in this context, for example.
- eip 8y ago
  If you think anyone 'really' wants to build your shitty CRUD app then you are deluding yourself. Do you 'really' love your job? Would you do it for free?
  - rrggrr 8y ago
    Ouch. So, I'm not in the software business, but either way the point of my Q&A is measure the distance between what an employee wants to during the day versus what probably needs to be done to accomplish the desired result. When the gap is big the hire/job never works out.
jfasi 8y ago
As someone who does both a lot of interviewing (at Google) and gives a lot of advice on interviewing to people just starting out, this rhymes with my intuition about why some people get into interviews: even without the use of brainteasers (which, for the record, are not used at Google), the interview process is hard. People rarely come out of it without at least a twinge of frustration of humiliation. I'm not immune to this, I got hired at Google despite having a few interviews where I thought "God damn that was horrible," and I didn't get offers at places I know I was good enough to work at because I wasn't performing at my best for whatever reason and nuked the interviews. I have a pet theory that when people interview they bring this buried frustration into the room with them and use the interviewing process to play the part of the people they feel humiliated them. It's conceivable that people with latent meanness, or as this abstract calls it tendencies for "narcissism and sadism" who aren't aware enough of their biases to try to counteract them, will use these sorts of questions to make themselves feel better. Also, and this is probably more specific, there's a bit of a mythos around companies like Google and friends that have tough interviews and high standards: they hire the best people, they're doing the best work, they're the smartest, etc. I can tell you from experience, that mythos doesn't hold up once you're doing you day to day job. I've seen people with stunning credentials doing what most would consider exciting work become genuinely bored. Perhaps for some of them interviewing is a way of reminding themselves of how desirable and high status their workplace is, to the detriment of the poor candidate. And of course, as a disclaimer, this isn't even anecdote, these are my own personal musings on psychology. Don't confuse this with insight into the Google hiring process.
- pinewurst 8y ago
  It's like military academy (or fraternity) hazing. You've been abused, therefore you want to pass the load of that abuse on. Maybe you delude yourself into calling it "tradition" or "high standards" or maybe you're just honest and enjoy the sadism of it all.
  - wolfgke 8y ago
    > It's like military academy (or fraternity) hazing. You've been abused, therefore you want to pass the load of that abuse on. I really cannot imagine why people have this behavior. I was often treated unfairly in life, so I try to architect systems that attempt to avoid my perceived unfairness so that the next generation does not have to suffer similarly (ideally in way for which correctness can be shown in a mathematically rigid way).
codingdave 8y ago
Interviewing others can certainly expose your own faults. In our last round of interviews, I certainly could have done better by our candidates. I learned, and will do better next time. But judging a company based on their questions may be expecting too much. Nobody is perfect, and if you expect perfection from someone who will end up being a member of your team, you might be setting everyone up for disappointment. Or you might be walking away from a good team or product just because they are not good at interviewing. Instead, if they ask questions that imply cultural problems or personality conflicts, ask about it. Find out if they are just bad interviewers, or if they do have deeper problems. Because bad interviewers are not always a bad sign -- it could mean they don't have to do it often. It could also mean they are trying to expand the skills of people new to interviewing by giving them opportunities to do something new. Also, companies who have mastered the interview process means they do it often, which could be a sign of high turnover. In short, it is good to notice problems, ask about them, then think through all the implications of the answers before making decisions.
bitL 8y ago
So, FAANG is full of carefully selected sadistic narcissists thinking they can "save the world"? Sounds like fun :-D Seriously, I thought brain teasers are a funny game and an intellectual version of "xyz" measuring contest, showcasing equivalent of "street smarts" in higher level brain functionality and a signal about how much boring one would be for an interviewer to work with. Now this secret dark-triad dimension completely bypassed my attention as I actually enjoyed them a lot.
yters 8y ago
It'd be interesting to see whether the interview brainteasers translate to job performance, especially where being able to solve lateral thinking and analytic problems is important for doing well at the job. That might explain why such questions are used during the job interview.
- solomatov 8y ago
  They don't according to this article: https://www.theatlantic.com/business/archive/2013/06/google-finally-admits-that-its-infamous-brainteasers-were-completely-useless-for-hiring/277053/ https://www.theatlantic.com/business/archive/2013/06/google-...
srinivgp 8y ago
The "brainteaser" question type in the survey is just another type of question. If you've seen/practiced Fermi estimates, it's pretty rote. If you've never seen/practiced Fermi estimates, it will sound like a horrible trick question. I wonder how well it would work to begin with "are you comfortable with Fermi estimates?". If so, go ahead and ask. If not, _teach_, do a couple together, then have the candidate do one on their own. The technique isn't hard, it's just surprising that it can work and work pretty well if you haven't already been exposed.
- hfdgiutdryg 8y ago
  "Fermi estimate" questions are stupid most of the time. Why? Because they leave out the educated part of educated guess. The idea is that inaccuracies in a series of educated guesses would tend to even out (though they're just as likely to compound). There is no such tendency in blind guesses, which is what most of these questions require.
- hyperpape 8y ago
  This might be a bit less sadistic, but I still think it's pointless. You have 30 minutes or an hour with a candidate. Why is this question the one that will tell you the most about whether they'd be a good hire?
puzzlerthrow 8y ago
jokes on them -- i'm way better at answering brainteaser questions than i am at coding on a whiteboard.
Nokinside 8y ago
Brainteasers are good for psychological evaluation. How does one perform under stress or when they are challenged in social situation. If the goal is to to find out what somebody can do, state a problem and start working together to solve it. For example ask them to help you to solve some problem and guide you trough solving it.
- cjg 8y ago
  Does the job involve high stress and a challenging social situation? Doesn't sounds like somewhere I want to work.
  - Nokinside 8y ago
    People who are mentally strong and competitive love working in that kid of environment. You need different recruiting and different 'back office' jobs for fragile people (60-70 percent of the population - and increasingly young males).
riyadparvez 8y ago
My issue is more with tricky algorithmic questions. I faced these tricky interview questions, where you either have to know the trick beforehand or there is NO way you can come up with a solution in an interview. I never understand the point of these questions. Because these esoteric hypothetical questions will almost never come up in real-life. They don't test someone's skills, because these problems don't follow any standard algorithmic problem solving procedure. They mostly test whether you know the trick or not, which is not a valid metric of skills. One famous example of this type of questions is to find loop in a singly connected linked list. Now almost everyone knows the trick. And thanks to that, interviewers have stopped asking questions like that.
- jf- 8y ago
  Most algorithms questions only prove one thing: whether a candidate has brushed up on algorithms beforehand. That has some usefulness, being a proxy measure for conscientiousness, but offers little beyond that. Maybe that they’re able to follow the reasoning, but it could just as easily be learned by rote and parroted back. The fact is that no one comes up with a from-scratch efficient algorithm for doing anything in 5 minutes, the candidate either knows it going in, or they’re screwed.
  - Timmah 8y ago
    I've been a cpp and java programmer for 12 years and not once needed to use algorithmic complexity on the job. The fact that so many places focus on this baffles me. Big-O questions should be reserved for system architect positions only. Out of all the arbitrary topics, I think they should focus on pedantic syntax questions for whatever language they use. Show snippets of code and ask "what's wrong with this picture?". That would at least select for people with better language mastery and, in theory, faster development times.
    - nicoburns 8y ago
      I don't really believe this. You use algorithmic complexity every time you choose between a HashMap and an Array. I'm a javascript developer, and I make that kind of choice several times a day. Having said that, I've never once needed to know the kinds of questions that the ask in these interviews off the top of my head. And I highly doubt google engineers need to often either.
nimbius 8y ago
I dont work in tech, im an engine mechanic by trade, but im fascinated to know why/how often this happens in a tech interview? Do programmers/sys admins really endure this kind of aggravation in their interviews? The hardest question I've ever asked one of my shop mechanics during an interview was how does Exhaust Gas Recirculation (EGR) in a Diesel engine affect thermal efficiency, smoke output and emissions of the engine...and that was for a full-time diesel tech at our cross town location. id imagine if i ever started throwing out manhole cover questions or underwater airplane bullshit, i'd catch some hands before I managed to hire anyone.
- pinewurst 8y ago
  As a long time tech person, it wasn't always like this at all. The brainteaser crap spread out from the likes of Microsoft, followed by the whiteboarding a little later.
- pnw_hazor 8y ago
  Think of the type of person that would suffer through this type of indignation. And, the type of person the gleefully puts people through this kind of hazing. Sounds like perfect formula for building toxic workplaces full of people with personality disorders.
  - pinewurst 8y ago
    That's why I'm not a software developer any more, even though I greatly enjoy the creative aspects of the work.
  - s73v3r_ 8y ago
    No, that's not entirely right. Many of us that hate it and don't want anything to do with it still endure it, because the alternative is not getting a job.
- Timmah 8y ago
  It happens because the types of problems software engineers face are random and require active engagement and creativity to solve. There's no book that you can read that says "if the program crashes, try these things". Yes there are basics but theres no real formula that always leads to the solution. Each program is unique enough that you have to study the internals of how it works and then test hypotheses about the root cause. In engines, they all follow the same predefined models. Software does have "models" but most people don't use them or they blend models together and don't document how they work. So we end up with having to solve many problems on the fly. Not everyone can do this (crucially, even people with cs degrees) , so to avoid hiring people who can't, they try to test for the ability during the interview. This is fraught with an entire slew of issues. Some of which are non-technical people asking the questions, applicants sharing the answers and gaming the system, or as the article says, sadistic interviewers that enjoy the feeling of power. To date, nobody has found a universally effective system to hire good programmers.
ElysianEagle 8y ago
I realize that the study was specifically talking about brainteasers, but I wonder if irrelevant algorithm questions have a similar motivation? I was once asked at an interview at a FAANG company to come up with an algo for finding the longest palindromic subsequence (!). At another I was asked to get all partitions of an integer (https://www.whitman.edu/mathematics/cgt_online/book/section03.03.html https://www.whitman.edu/mathematics/cgt_online/book/section0...). I'm pretty certain I'd never be working on problems of that nature as part of the role.
- repsilat 8y ago
  > an algo for finding the longest palindromic subsequence (!) Not irrelevant to my professional career, to be honest. There have been a few situations where being able to see a tricky, unfamiliar problem and after some thought say, "oh, we can use a DP algo for that" has saved a large amount of future headache. Only a small fraction of people on the team need to have that skill (more in some teams than others, too) so I don't think it's a great thing to test for (much less require), but it's actually a pretty valuable skill in my experience.
inetknght 8y ago
How do I view the PDF with my javascript-disabled browser?
- teraflop 8y ago
  According to the sidebar, you have to pay $38 to download the PDF.
  - inetknght 8y ago
    That sidebar also doesn't load if javascript is disabled.
  - jimmies 8y ago
    Or you can `/s/doi.org/sci-hub.tw/` the doi URL. ;-)
masto 8y ago
I thought everybody stopped doing this.
EngineerBetter 8y ago
We ask people to pair-program with an interviewer on a realistic task. No tricks, no brainteasers. We just get to see how people do the job we want them to do.
- biztos 8y ago
  Is pair-programming actually used in your company? Just curious, as I do something similar but solely to see how the person writes code. Folks are free to pair-program or not once hired. I've found almost everyone needs help on some simple part of the task, and how they respond to the help is pretty useful information.
  - bitwize 8y ago
    I've interviewed at places with a pair programming part of the interview, that didn't use pairing during actual development. It is an enormous help toward glimpsing the mind of the programmer as they program.
  - EngineerBetter 8y ago
    Yep, we pair at least 80% of the time.
manish_gill 8y ago
I took an interview today and was asked a simple problem which I immediately answered with linear time complexity. The interviewer then went on to say "no I'm looking for a specific answer which involves a trick". Literally said those words. Eventually figured it out but I came away with a bad impression of the company. Not the kind of stuff I judge people on.
- dbt00 8y ago
  “Are most of your day to day problems solved by rote memorization of esoterica?”
- Timmah 8y ago
  "Well, you're not hiring a trickster."
- tjungblut 8y ago
  would you be able to share the question?
- kitanata 8y ago
  I had a similar thing happen when interviewing for ShutterStock 2-3 years ago. Wanted me to implement fibonaccci sequencer. I of course, forgot the mathematical definition of Fibonancii sequence, so went to Wikipedia to look it up. Was told I couldn’t use outside resources, then was told to think through what that sequence does and struggled a lot. Finally got through it with a really terrible recursive implementation, because the interviewer kept interrupting me and correcting me over the course of the interview, which took about an hour. I asked the interview where they put my skill level and I shit you not they said entry level to junior. I’ve been programming for 10 years at that point, and was significantly qualified for the position. After the interview was over, looked up a constant time formula based on the Golden Mean (from Wikipedia) and implemented a constant time algorithm to solve it. Took 4 lines of code and 10 minutes to do (including a test suite) Sent it in via email, and basically told them why their hiring process is broken and insulting to candidates. I didn’t get the job, thank god.
ccvannorman 8y ago
I have a totally different sentiment to asking and receiving these wuestions. To me it's playful and fun and can establish how somone thinks and communicates. Asking questions like these I might do with my friends just for fun
Syzygies 8y ago
I interviewed a full day at a mathematical hedge fund. Not exactly brainteasers, rather a barrage of good, solid math and stats questions. Such a day is exhilarating (like skiing the Olympics) but truly exhausting, and only partially related to whether one gets an offer. My advice to anyone doing this would be to give up the last 10% of performance in order to focus some on contextual awareness. These puzzles are a form of pre-surgical anesthesia, before they ask the real questions. You need the spare cycles at the end of a long day to imagine what casual conversation might really be exploring, how other applicants might misread the subtext, what mistakes in previous hiring they're trying not to repeat. One could be a genius trained seal, by buying into the brainteaser game, and miss the broader question of how you will fit into their firm. One who sees broader contexts ends up running places; don't make it clear that's not you.
- throwawaymath 8y ago
  Would you say the questions that fund asked you are relevant to the work you do (or would be doing) there? I think that's the core criticism most people levy against the "high tech" interview questions.
  - segmondy 8y ago
    People that don't get the offer tend to complain about such things. They are asking questions to measure how you think, to measure your potential, to see how you handle stress. If you are stressing over an interview, what will you do when you lose $1 million in 10 seconds?
    - throwawaymath 8y ago
      Sure I get the gist, I've heard this reasoning quite a lot and I'm not condoning or condemning it. I was specifically interested in hearing if the OP thought the math they asked him about was relevant to the math actually done on the job. In my experience, it isn't really (except insofar as it might be the same field, from a 30,000 foot view).
    - Timmah 8y ago
deleted 8y ago
[deleted]
framebit 8y ago
I had one of these brainteaser questions in an interview once, and I liked my mathematician friend’s answer to it the best. Q: Given infinite setup time and infinite money, how would you fill the inside of a 747 with ping pong balls as quickly as possible? A: Define the inside of the plane as the exterior. Now, by definition, all ping pong balls are inside the plane. QED.
- jgtrosh 8y ago
  If there's a specific number of balls, that's a great answer; if you need to fill up the interior, you're gonna need a lot more balls!!
- RodgerTheGreat 8y ago
  All ping pong balls may now be "inside" the plane, given this approach, but he's done a horrendously poor job of filling the plane.
  - pmiller2 8y ago
    Not really. It is as full as possible. :)
- bfrydl 8y ago
  My answer: I use my infinite resources to become benevolent dictator of Earth. With my endless wealth I prevent climate change, provide the basic needs of every person on the planet, educate the world, and set up a global system of sustainable reproduction. I have infinite resources so I assume it's not eugenics or something dystopian, but more a collective human understanding of responsibility. Finally, with our future ensured, I dedicate as many of the world's top minds as possible to preserving my own life at any cost. Once I have achieved immortality, likely by transplanting my mind into a robot body that can be updated over time so I remain the most advanced being on the planet, I steer the course of human history far into the future. All human suffering is ended. We all work as one to better ourselves. We spend decades, centuries, millennia improving not only our society, but our technology for inserting ping pong balls into 747s. Under my kind leadership as an immortal god-king of increasingly sophisticated artificial intelligence, humanity becomes the dominant species in a surprisingly crowded universe. With each new species we meet, we find new allies in our advancement. Entire galaxies unite under my rule and never-ending resources, each furthering my transcendence of life and the speed at which we can put small plastic balls into extremely archaic public transportation. Finally, at the far end of time, when there is nothing left to learn, no secret of the universe left unknown to me—I am now a single bio-mechanical entity composed of all intelligent life in existence—my setup is complete. Using technology so advanced that it cannot be described by words we have so far invented, technology so unimaginable it exceeds the power we ascribe to magic, I fill the plane. It is instantaneous. There is no time during which the plane is being filled. There is only before and after it was filled. The creator of the universe descends on its final moments. The being sees the plane. It weeps for the beauty its children have created.
tw1010 8y ago
Woah, papers on "dark motives", that sounds interesting. Anyone know of any other good papers on hidden motives similar to this one?
- arayh 8y ago
  I like this paper on dark patterns: http://colingray.me/wp-content/uploads/2018_Grayetal_CHI_DarkPatternsUXDesign.pdf http://colingray.me/wp-content/uploads/2018_Grayetal_CHI_Dar...
emodendroket 8y ago
I honestly expected this to be talking about discriminatory use of these questions, so I'm not sure if I should be relieved to see that's not what it's about.
mfonda 8y ago
Being able to approximately estimate numbers is an excellent skill for software developers to have. Imagine you're running some operation that affects N things or has N results. After seeing the results, it's very helpful to ask yourself "Does N roughly match what my intuition/estimate told me it would be?" If your estimate is off by at least an order of magnitude, it's a hint there could be a bug in your solution and that it warrants a closer look. I don't ask brainteaser type questions, but I can see their appeal given the value of estimates. I also don't see why they're so universally hated. Is doing back-of-the-napkin math something a lot of people struggle with?
jkingsbery 8y ago
I wonder how the "guess how many dog leashes there are in Kansas" kind of questions would compare to more job-relevant estimation interview questions - i.e., would the paper find the same sort of conclusions?
spitfire 8y ago
Hunter and Schmidt did a meta-study of 85 (now 100, with the follow up) years of research on hiring criteria. [1] There are three attributes you need to select for to identify performing employees in intellectual fields. - General mental ability (Are they generally smart) Use WAIS or if there are artifacts of GMA(Complex work they've done themselves) available use them as proxies. Using IQ is effectively illegal[2] in the US, so you'll have to find a test that acts as a good proxy. - Work sample test. NOT HAZING! As close as possible to the actual work they'd be doing. Try to make it apples-to-apples comparison across candidates. Also, try and make accomidations for candidates not knowing your company shibboleth/tools/infra. - Integrity. The first two won't matter if you hire dishonest people or politicians. There are existing tests available for this, you can purchase for < $50 per use. This alone will get you > 65% hit rate [1], and can be done inside of three hours. There's no need for day long (or multi-day) gladiator style gauntlets. Apply this process to EVERYONE, including that elite cool kid from the "right" school/company. You don't want to exclude part of your sample population! [1] http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%.. http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%.... [2] Technically IQ tests are not "illegal", but the bar courts have decided companies have to climb is so high it effectively means they are. There is existing case law directly covering IQ tests in hiring. You should speak with your lawyer before you decide to try IQ tests. [3] 2016 update of Hunter Schmidt: https://home.ubalt.edu/tmitch/645/articles/2016-100%20Yrs%20Working%20Paper%20for%20Research%20Gate%2010-17.pdf https://home.ubalt.edu/tmitch/645/articles/2016-100%20Yrs%20...
- pmiller2 8y ago
  What sort of meaningful work sample test can you do in 3 hours?
  - RhodesianHunter 8y ago
    "Here is the almost complete skeleton of a project in your preferred language (presumably, the one you're applying to use full time), finish it by implementing this vital piece of business logic."
  - Izkata 8y ago
quotemstr 8y ago
Social science generally doesn't reproduce. Why should anyone pay attention to this story? It seems like it plays into the aptitude-tests-are-bad narrative, and I fundamentally disagree with that idea.
opportune 8y ago
I guess I’m evil because I would refer working with people who can pass brain teasers to those who can’t. The problem is that you can “prepare” for brain teaser questions, so you end up selecting for people who prepare more than people who are actually smart.
8_hours_ago 8y ago
I recently interviewed at Apple and one interviewer caused me to decide to never work there. I was put into a conference room and had 8 people over 6 hours come in and interview me one after another. Each person asked me about my background and then jumped into a whiteboard coding problem. Most of the interviewers were understanding of the fact that writing code on a whiteboard is nothing like writing code on a computer, and were helpful in pointing out simple mistakes that a compiler would have caught. While writing code for the last interviewer I had to do some simple division. I came up with the wrong answer and told the interviewer that I wasn't sure if that was correct, and to let me know if it was incorrect. He blankly stared at me and said he wouldn't help me. Usually I would be able to do the division in my head, but after 6 hours of constant interviewing my brain was starting to turn into mush. I went on and finished the solution and he said that there was an issue but wouldn't tell me where. I asked if it was an issue with the division and he said yes. He still wouldn't tell me the answer, so I floundered for a bit while trying to remember how to do long division. At one point I turned to him and said "I'm sorry this is taking me so long, it's been a very long day" and he replied "sometimes you will have to fix your code on the spot". It seemed like he was trying to see if I would crack under the pressure. Eventually I came up with the correct answer and was able to finish the problem, but it completely drained me. I decided then that I never wanted to work with that person, or with other people like them. Some jobs require performance under pressure, but programming is not one of them. The only thing that the interviewer was evaluating was my ability to do math problems in my head after being interviewed for 6 hours. Don't they have calculators at Apple? Or phones, perhaps?
- corey_moncure 8y ago
  So you met 8 people, and one of them was a jerk? I'd say that's a pretty acceptable jerk coefficient for a workplace. I've been in places north of 0.5.
  - whichdan 8y ago
    If 12.5% of people in an office were jerks, I'd consider that a very toxic culture.
  - 8y ago
cubano 8y ago
Maybe the correct stratagy is turn the tables and try to lighten the mood of the interview if a brainteaser is asked, and sort of shuck "oh one of these things? (laugh)...ok let's work this out together" and proceed to try to engage the interviewer with estimating the number of floors in an average building, the number of windows per story, and the number of buildings, all the time asking the interviewer what his estimates would be? Like anything to prevent yourself from locking up and taking the damn thing too fucking seriously. The reality is we are all going to interview, they are going to mostly suck, and you will almost certainly eventually be hired somewhere.
- jpttsn 8y ago
  The interviewer doesn’t need to know the number of windows. They need to know that you will take things seriously on demand.
  - ubernostrum 8y ago
    The candidate doesn't need to know how to estimate the number of windows. They need to know you're the sort of interviewer who likes to be the organ grinder while everyone else is the dancing monkey. Many common interview questions give this away quickly, which is useful.
ec664 8y ago
[no paywall] Link to full paper: https://sites01.lsu.edu/faculty/zhanglab/wp-content/uploads/sites/158/2017/05/apps_12163_Rev.pdf https://sites01.lsu.edu/faculty/zhanglab/wp-content/uploads/...
justfor1comment 8y ago
To account for some interviewers being sadist in general, may be companies can implement a per interviewer rejection quota. For example, if you interview 10 candidates at max you can reject 7 candidates. You have to yes to 3 candidates or be removed from the interviewing committee. This will ensure that just one bitter person is not preventing good candidates from clearing the interview process.
- pjlegato 8y ago
  How about Olympics style scoring -- a panel of interviewers rate each candidate, then the highest and lowest individual scores are removed before averaging?
- emidln 8y ago
  I would never vote for a candidate I didn't believe in. If you want to remove qualified engineers from hiring decisions based on arbitrary metrics, I'd probably be looking for another job on the side.
- ubernostrum 8y ago
  I like this video: https://www.youtube.com/watch?v=r8RxkpUvxK0 https://www.youtube.com/watch?v=r8RxkpUvxK0 The speaker formerly worked at Google, and around 8:55 in the video relates the story of being on a hiring committee that was known for being tough on candidates. And a recruiter once forwarded a set of interview packets to them, and they voted "no hire" on all of them. The recruiter then revealed to the committee that they'd been given their own packets, slightly anonymized, from when they were interviewed, and had just voted against hiring themselves.
m3kw9 8y ago
For me a brain teaser means the company can’t get to the point and unknowingly is just beating around the bush.
anonytrary 8y ago
A Fermi problem is not a brain teaser in the colloquial sense, it's more of an approximation game. If you looked up an answer a week ago and memorized it, then regurgitated it, you'd still get the question "wrong". No one expects you to come up with the right number of windows, they expect you to get somewhere plausible from scratch. Asking someone how many windows are in NYC is not fundamentally different than asking someone how many blades of grass are on a soccer field. You can know almost nothing and still arrive at the correct order of magnitude -- all it tests is someone's ability to do napkin math and break open-ended problems into manageable parts. Is this the right thing to test? That depends on what you're hiring someone for.
ummonk 8y ago
To me, "Estimate how many windows are in New York" is a much more pleasant question to answer than "Are you a good listener?" or "Tell me about a time when you failed"...
- sgustard 8y ago
  When you're done hiring you've surrounded yourself with people who are good at estimating urban fenestration, but are poor listeners who blame others when they fail.
  - ummonk 8y ago
    I'm not saying that is the best question to ask an interviewee. I'm just saying it's the nicest of the three for me to be on the receiving end of. (Although, to be clear, "are you a good listener?" is an absolutely terrible question.)
logifail 8y ago
I was asked a series of brainteaser interviews at a university interview. It was immediately clear that the purpose of the questions wasn't really to see if I knew the answers, but to see how I responded when asked about something I didn't (immediately) know the answer to. Isn't this a perfectly legitimate way to try to see how a candidate might deal with solving problems they don't know the answers to? Ironically one of the interviewers (a somewhat crusty old astrophysics professor) later became my physics tutor, and continued to ask us brainteaser questions during tutorial sessions. For instance, I dimly recall us being asked to "estimate the change in the length of the Earth's day if all the cars in the UK were to drive on the right-hand side of the road instead of on the left"...
est 8y ago
I think it's only fair if interviewee was allowed to ask a counter brainteaser question. Let the both sides suffer some kind of stress.
ectospheno 8y ago
When interviewing I would often ask one (and only one) question of this type. The rest of the interview was your standard set of questions to see what type of person you were sitting across from. The one "brainteaser" question was "Estimate how fast human hair grows in miles per hour. Order of magnitude is fine." I just wanted to see that they not only knew how to estimate something but that they understood this particular estimation wasn't important given the job they were interviewing for and could thus be answered quickly rather than accurately. The one person who answered immediately was off by 3 orders of magnitude but he was the best hire we had. He could spot BS a mile away.
csense 8y ago
I think this particular question (a so-called Fermi problem) is a perfectly legitimate interview question. Being able to do "back-of-the-envelope calculations" is a necessary or helpful skill for many technical and analytical jobs. It's not about getting the right answer, it's about the candidate's process to arrive at the answer.

290 points by 33degrees 8y ago

This reminds me of Jewish Problems[0], designed to "prevent Jews and other undesirables from getting a passing grade" at the entrance exams to the math department of Moscow State University. [0] https://arxiv.org/abs/1110.1556 https://arxiv.org/abs/1110.1556

golergka 8y ago
Wow, I never knew they actually published a paper on it. Coming from a Moscow math school that traditionally had a lot of jewish students, most of whom went on to MSU to study math, I remember studying these exact problems - although in my time, this filter was supposed to long been gone.
hogwild 8y ago
For a second there I thought you were talking about the interview process for becoming a Jew when they weed out undesirables.
markatkinson 8y ago
This was discussed in Edward Frenkels book Love and Math
taurath 8y ago
As a self-taught programmer that has had to go back to get "cs fundamentals", half of interview questions feel like that - if you just know about x or y data structure for this particular edge case, everything becomes simple/easy.

diN0bot 8y ago

interesting that “teasers” are viewed as callous and abusive and yet analytical problem solving is viewed as predictive. fine line.

frandroid 8y ago
A brain teaser usually has nothing to do with the work... What would asking about the shape of manhole covers have to do with OOP?
- darawk 8y ago
  The theory would be that it allows the interviewee to demonstrate abstract reasoning skills, which are useful in many domains.
  - prolepunk 8y ago
    Also the interviewee needs to have exactly the same assumptions about the world as interviewer (or whoever come up with the brainteaser) and the knowledge of the same basic facts. I mean how do you even approach questions like -- how many snowflakes on the south pole? how many fish in all of the oceans? how much grass is in Mongolia?
    - darawk 8y ago
      I think that's the point of the question: to see how you approach it. The 'correct' approach that they'd be looking for is 'Fermi estimation': https://en.wikipedia.org/wiki/Fermi_problem https://en.wikipedia.org/wiki/Fermi_problem
      - ubernostrum 8y ago
        No, the point is to try to rationalize cargo-culting. Fermi did this, and Fermi was brilliant, so if we require candidates to do this we'll end up hiring people who are as brilliant as Fermi! Here's a brainteaser: spot all the flaws in that chain of "reasoning".
        darawk 8y ago
        Um, no. It's not cargo-culting. It's the correct way of estimating things. It has nothing to do with Fermi having done it. Some things actually have correct answers. 2 + 2 = 4, is not cargo culting.

rossdavidh 8y ago

So, there's a hidden value to these sorts of questions, but it is useful to the interviewee, not the interviewer. If they use lots of these questions, you probably don't want to work there, or at least not for the person who's asking those questions.

welcome_dragon 8y ago
I disagree, as long as getting the right answer isn't important. I flat out tell people I don't care if they get the right answer, that I just want to hear their thought process. Such questions used in this way are extremely valuable.
- athenot 8y ago
  I do the same thing. It's how the thought process works that's interesting, not the result. There is no problem or teaser whose answer is both short enough to be devised during an interview AND is representative of how a candidate would perform in real projects. Our field deals with complex systems in many dimensions. Instead, looking for how a candidate approaches a problem, do they just go for the obvious solutions, how creative will they go, any experience they can draw on, how well they overcome unknowns, etc.
  - gaius 8y ago
    It's how the thought process works that's interesting, not the result. I wonder how many interviewers have the necessary background in clinical psychiatry to analyze a candidate's thought process, or if that's just more handwaving away hazing.
- nomel 8y ago
  I think a good compromise is to modify the questions so it's actually applicable to the context of the job.
- sidlls 8y ago
  They're valuable for what? Showing off a thought process in a high pressure one hour meeting with a judge watching you?

rrggrr 8y ago

The single most effective interview question I ask: "What do you want to do?". I drill down on that to a granulator level as I try to make sure the tasks, duties and overall position matches what the candidate really wants for themselves. Its followed-up with a "How would you do it?", again persisting in the discussion (not interrogation) until we both are comfortable with the sensibility and symmetry of the answers.

wonderbear 8y ago
I'm going to try this. The questions my team comes up with haven't been very satisfying and this seems like a good way to find out what they do know vs what they don't.
- rrggrr 8y ago
  I learned this from one of the best recruiters in finance. I can't take credit for it.
pinewurst 8y ago
That may be a good conversational start but I'm less certain that anyone really knows at any deep granularity what they want to do (professionally). There are a lot of roads to autonomy, mastery and purpose, and a lot of speed traps/bad exits. My current job, which I've just resigned because it became so awful, was on the surface what I wanted to do, until it changed. It would be difficult to discuss the before and after in this context, for example.
eip 8y ago
If you think anyone 'really' wants to build your shitty CRUD app then you are deluding yourself. Do you 'really' love your job? Would you do it for free?
- rrggrr 8y ago
  Ouch. So, I'm not in the software business, but either way the point of my Q&A is measure the distance between what an employee wants to during the day versus what probably needs to be done to accomplish the desired result. When the gap is big the hire/job never works out.

jfasi 8y ago

As someone who does both a lot of interviewing (at Google) and gives a lot of advice on interviewing to people just starting out, this rhymes with my intuition about why some people get into interviews: even without the use of brainteasers (which, for the record, are not used at Google), the interview process is hard. People rarely come out of it without at least a twinge of frustration of humiliation. I'm not immune to this, I got hired at Google despite having a few interviews where I thought "God damn that was horrible," and I didn't get offers at places I know I was good enough to work at because I wasn't performing at my best for whatever reason and nuked the interviews. I have a pet theory that when people interview they bring this buried frustration into the room with them and use the interviewing process to play the part of the people they feel humiliated them. It's conceivable that people with latent meanness, or as this abstract calls it tendencies for "narcissism and sadism" who aren't aware enough of their biases to try to counteract them, will use these sorts of questions to make themselves feel better. Also, and this is probably more specific, there's a bit of a mythos around companies like Google and friends that have tough interviews and high standards: they hire the best people, they're doing the best work, they're the smartest, etc. I can tell you from experience, that mythos doesn't hold up once you're doing you day to day job. I've seen people with stunning credentials doing what most would consider exciting work become genuinely bored. Perhaps for some of them interviewing is a way of reminding themselves of how desirable and high status their workplace is, to the detriment of the poor candidate. And of course, as a disclaimer, this isn't even anecdote, these are my own personal musings on psychology. Don't confuse this with insight into the Google hiring process.

pinewurst 8y ago
It's like military academy (or fraternity) hazing. You've been abused, therefore you want to pass the load of that abuse on. Maybe you delude yourself into calling it "tradition" or "high standards" or maybe you're just honest and enjoy the sadism of it all.
- wolfgke 8y ago
  > It's like military academy (or fraternity) hazing. You've been abused, therefore you want to pass the load of that abuse on. I really cannot imagine why people have this behavior. I was often treated unfairly in life, so I try to architect systems that attempt to avoid my perceived unfairness so that the next generation does not have to suffer similarly (ideally in way for which correctness can be shown in a mathematically rigid way).

codingdave 8y ago

Interviewing others can certainly expose your own faults. In our last round of interviews, I certainly could have done better by our candidates. I learned, and will do better next time. But judging a company based on their questions may be expecting too much. Nobody is perfect, and if you expect perfection from someone who will end up being a member of your team, you might be setting everyone up for disappointment. Or you might be walking away from a good team or product just because they are not good at interviewing. Instead, if they ask questions that imply cultural problems or personality conflicts, ask about it. Find out if they are just bad interviewers, or if they do have deeper problems. Because bad interviewers are not always a bad sign -- it could mean they don't have to do it often. It could also mean they are trying to expand the skills of people new to interviewing by giving them opportunities to do something new. Also, companies who have mastered the interview process means they do it often, which could be a sign of high turnover. In short, it is good to notice problems, ask about them, then think through all the implications of the answers before making decisions.

bitL 8y ago

So, FAANG is full of carefully selected sadistic narcissists thinking they can "save the world"? Sounds like fun :-D Seriously, I thought brain teasers are a funny game and an intellectual version of "xyz" measuring contest, showcasing equivalent of "street smarts" in higher level brain functionality and a signal about how much boring one would be for an interviewer to work with. Now this secret dark-triad dimension completely bypassed my attention as I actually enjoyed them a lot.

yters 8y ago

It'd be interesting to see whether the interview brainteasers translate to job performance, especially where being able to solve lateral thinking and analytic problems is important for doing well at the job. That might explain why such questions are used during the job interview.

solomatov 8y ago
They don't according to this article: https://www.theatlantic.com/business/archive/2013/06/google-finally-admits-that-its-infamous-brainteasers-were-completely-useless-for-hiring/277053/ https://www.theatlantic.com/business/archive/2013/06/google-...

srinivgp 8y ago

The "brainteaser" question type in the survey is just another type of question. If you've seen/practiced Fermi estimates, it's pretty rote. If you've never seen/practiced Fermi estimates, it will sound like a horrible trick question. I wonder how well it would work to begin with "are you comfortable with Fermi estimates?". If so, go ahead and ask. If not, _teach_, do a couple together, then have the candidate do one on their own. The technique isn't hard, it's just surprising that it can work and work pretty well if you haven't already been exposed.

hfdgiutdryg 8y ago
"Fermi estimate" questions are stupid most of the time. Why? Because they leave out the educated part of educated guess. The idea is that inaccuracies in a series of educated guesses would tend to even out (though they're just as likely to compound). There is no such tendency in blind guesses, which is what most of these questions require.
hyperpape 8y ago
This might be a bit less sadistic, but I still think it's pointless. You have 30 minutes or an hour with a candidate. Why is this question the one that will tell you the most about whether they'd be a good hire?

puzzlerthrow 8y ago

jokes on them -- i'm way better at answering brainteaser questions than i am at coding on a whiteboard.

Nokinside 8y ago

Brainteasers are good for psychological evaluation. How does one perform under stress or when they are challenged in social situation. If the goal is to to find out what somebody can do, state a problem and start working together to solve it. For example ask them to help you to solve some problem and guide you trough solving it.

cjg 8y ago
Does the job involve high stress and a challenging social situation? Doesn't sounds like somewhere I want to work.
- Nokinside 8y ago
  People who are mentally strong and competitive love working in that kid of environment. You need different recruiting and different 'back office' jobs for fragile people (60-70 percent of the population - and increasingly young males).

riyadparvez 8y ago

My issue is more with tricky algorithmic questions. I faced these tricky interview questions, where you either have to know the trick beforehand or there is NO way you can come up with a solution in an interview. I never understand the point of these questions. Because these esoteric hypothetical questions will almost never come up in real-life. They don't test someone's skills, because these problems don't follow any standard algorithmic problem solving procedure. They mostly test whether you know the trick or not, which is not a valid metric of skills. One famous example of this type of questions is to find loop in a singly connected linked list. Now almost everyone knows the trick. And thanks to that, interviewers have stopped asking questions like that.

jf- 8y ago
Most algorithms questions only prove one thing: whether a candidate has brushed up on algorithms beforehand. That has some usefulness, being a proxy measure for conscientiousness, but offers little beyond that. Maybe that they’re able to follow the reasoning, but it could just as easily be learned by rote and parroted back. The fact is that no one comes up with a from-scratch efficient algorithm for doing anything in 5 minutes, the candidate either knows it going in, or they’re screwed.
- Timmah 8y ago
  I've been a cpp and java programmer for 12 years and not once needed to use algorithmic complexity on the job. The fact that so many places focus on this baffles me. Big-O questions should be reserved for system architect positions only. Out of all the arbitrary topics, I think they should focus on pedantic syntax questions for whatever language they use. Show snippets of code and ask "what's wrong with this picture?". That would at least select for people with better language mastery and, in theory, faster development times.
  - nicoburns 8y ago
    I don't really believe this. You use algorithmic complexity every time you choose between a HashMap and an Array. I'm a javascript developer, and I make that kind of choice several times a day. Having said that, I've never once needed to know the kinds of questions that the ask in these interviews off the top of my head. And I highly doubt google engineers need to often either.

nimbius 8y ago

I dont work in tech, im an engine mechanic by trade, but im fascinated to know why/how often this happens in a tech interview? Do programmers/sys admins really endure this kind of aggravation in their interviews? The hardest question I've ever asked one of my shop mechanics during an interview was how does Exhaust Gas Recirculation (EGR) in a Diesel engine affect thermal efficiency, smoke output and emissions of the engine...and that was for a full-time diesel tech at our cross town location. id imagine if i ever started throwing out manhole cover questions or underwater airplane bullshit, i'd catch some hands before I managed to hire anyone.

pinewurst 8y ago
As a long time tech person, it wasn't always like this at all. The brainteaser crap spread out from the likes of Microsoft, followed by the whiteboarding a little later.
pnw_hazor 8y ago
Think of the type of person that would suffer through this type of indignation. And, the type of person the gleefully puts people through this kind of hazing. Sounds like perfect formula for building toxic workplaces full of people with personality disorders.
- pinewurst 8y ago
  That's why I'm not a software developer any more, even though I greatly enjoy the creative aspects of the work.
- s73v3r_ 8y ago
  No, that's not entirely right. Many of us that hate it and don't want anything to do with it still endure it, because the alternative is not getting a job.
Timmah 8y ago
It happens because the types of problems software engineers face are random and require active engagement and creativity to solve. There's no book that you can read that says "if the program crashes, try these things". Yes there are basics but theres no real formula that always leads to the solution. Each program is unique enough that you have to study the internals of how it works and then test hypotheses about the root cause. In engines, they all follow the same predefined models. Software does have "models" but most people don't use them or they blend models together and don't document how they work. So we end up with having to solve many problems on the fly. Not everyone can do this (crucially, even people with cs degrees) , so to avoid hiring people who can't, they try to test for the ability during the interview. This is fraught with an entire slew of issues. Some of which are non-technical people asking the questions, applicants sharing the answers and gaming the system, or as the article says, sadistic interviewers that enjoy the feeling of power. To date, nobody has found a universally effective system to hire good programmers.

ElysianEagle 8y ago

I realize that the study was specifically talking about brainteasers, but I wonder if irrelevant algorithm questions have a similar motivation? I was once asked at an interview at a FAANG company to come up with an algo for finding the longest palindromic subsequence (!). At another I was asked to get all partitions of an integer (https://www.whitman.edu/mathematics/cgt_online/book/section03.03.html https://www.whitman.edu/mathematics/cgt_online/book/section0...). I'm pretty certain I'd never be working on problems of that nature as part of the role.

repsilat 8y ago
> an algo for finding the longest palindromic subsequence (!) Not irrelevant to my professional career, to be honest. There have been a few situations where being able to see a tricky, unfamiliar problem and after some thought say, "oh, we can use a DP algo for that" has saved a large amount of future headache. Only a small fraction of people on the team need to have that skill (more in some teams than others, too) so I don't think it's a great thing to test for (much less require), but it's actually a pretty valuable skill in my experience.

inetknght 8y ago

How do I view the PDF with my javascript-disabled browser?

teraflop 8y ago
According to the sidebar, you have to pay $38 to download the PDF.
- inetknght 8y ago
  That sidebar also doesn't load if javascript is disabled.
- jimmies 8y ago
  Or you can `/s/doi.org/sci-hub.tw/` the doi URL. ;-)

masto 8y ago

I thought everybody stopped doing this.

EngineerBetter 8y ago

We ask people to pair-program with an interviewer on a realistic task. No tricks, no brainteasers. We just get to see how people do the job we want them to do.

biztos 8y ago
Is pair-programming actually used in your company? Just curious, as I do something similar but solely to see how the person writes code. Folks are free to pair-program or not once hired. I've found almost everyone needs help on some simple part of the task, and how they respond to the help is pretty useful information.
- bitwize 8y ago
  I've interviewed at places with a pair programming part of the interview, that didn't use pairing during actual development. It is an enormous help toward glimpsing the mind of the programmer as they program.
- EngineerBetter 8y ago
  Yep, we pair at least 80% of the time.

manish_gill 8y ago

I took an interview today and was asked a simple problem which I immediately answered with linear time complexity. The interviewer then went on to say "no I'm looking for a specific answer which involves a trick". Literally said those words. Eventually figured it out but I came away with a bad impression of the company. Not the kind of stuff I judge people on.

dbt00 8y ago
“Are most of your day to day problems solved by rote memorization of esoterica?”
Timmah 8y ago
"Well, you're not hiring a trickster."
tjungblut 8y ago
would you be able to share the question?
kitanata 8y ago
I had a similar thing happen when interviewing for ShutterStock 2-3 years ago. Wanted me to implement fibonaccci sequencer. I of course, forgot the mathematical definition of Fibonancii sequence, so went to Wikipedia to look it up. Was told I couldn’t use outside resources, then was told to think through what that sequence does and struggled a lot. Finally got through it with a really terrible recursive implementation, because the interviewer kept interrupting me and correcting me over the course of the interview, which took about an hour. I asked the interview where they put my skill level and I shit you not they said entry level to junior. I’ve been programming for 10 years at that point, and was significantly qualified for the position. After the interview was over, looked up a constant time formula based on the Golden Mean (from Wikipedia) and implemented a constant time algorithm to solve it. Took 4 lines of code and 10 minutes to do (including a test suite) Sent it in via email, and basically told them why their hiring process is broken and insulting to candidates. I didn’t get the job, thank god.

ccvannorman 8y ago

I have a totally different sentiment to asking and receiving these wuestions. To me it's playful and fun and can establish how somone thinks and communicates. Asking questions like these I might do with my friends just for fun

Syzygies 8y ago

I interviewed a full day at a mathematical hedge fund. Not exactly brainteasers, rather a barrage of good, solid math and stats questions. Such a day is exhilarating (like skiing the Olympics) but truly exhausting, and only partially related to whether one gets an offer. My advice to anyone doing this would be to give up the last 10% of performance in order to focus some on contextual awareness. These puzzles are a form of pre-surgical anesthesia, before they ask the real questions. You need the spare cycles at the end of a long day to imagine what casual conversation might really be exploring, how other applicants might misread the subtext, what mistakes in previous hiring they're trying not to repeat. One could be a genius trained seal, by buying into the brainteaser game, and miss the broader question of how you will fit into their firm. One who sees broader contexts ends up running places; don't make it clear that's not you.

throwawaymath 8y ago
Would you say the questions that fund asked you are relevant to the work you do (or would be doing) there? I think that's the core criticism most people levy against the "high tech" interview questions.
- segmondy 8y ago
  People that don't get the offer tend to complain about such things. They are asking questions to measure how you think, to measure your potential, to see how you handle stress. If you are stressing over an interview, what will you do when you lose $1 million in 10 seconds?
  - throwawaymath 8y ago
    Sure I get the gist, I've heard this reasoning quite a lot and I'm not condoning or condemning it. I was specifically interested in hearing if the OP thought the math they asked him about was relevant to the math actually done on the job. In my experience, it isn't really (except insofar as it might be the same field, from a 30,000 foot view).
  - Timmah 8y ago

deleted 8y ago

[deleted]

framebit 8y ago

I had one of these brainteaser questions in an interview once, and I liked my mathematician friend’s answer to it the best. Q: Given infinite setup time and infinite money, how would you fill the inside of a 747 with ping pong balls as quickly as possible? A: Define the inside of the plane as the exterior. Now, by definition, all ping pong balls are inside the plane. QED.

jgtrosh 8y ago
If there's a specific number of balls, that's a great answer; if you need to fill up the interior, you're gonna need a lot more balls!!
RodgerTheGreat 8y ago
All ping pong balls may now be "inside" the plane, given this approach, but he's done a horrendously poor job of filling the plane.
- pmiller2 8y ago
  Not really. It is as full as possible. :)
bfrydl 8y ago
My answer: I use my infinite resources to become benevolent dictator of Earth. With my endless wealth I prevent climate change, provide the basic needs of every person on the planet, educate the world, and set up a global system of sustainable reproduction. I have infinite resources so I assume it's not eugenics or something dystopian, but more a collective human understanding of responsibility. Finally, with our future ensured, I dedicate as many of the world's top minds as possible to preserving my own life at any cost. Once I have achieved immortality, likely by transplanting my mind into a robot body that can be updated over time so I remain the most advanced being on the planet, I steer the course of human history far into the future. All human suffering is ended. We all work as one to better ourselves. We spend decades, centuries, millennia improving not only our society, but our technology for inserting ping pong balls into 747s. Under my kind leadership as an immortal god-king of increasingly sophisticated artificial intelligence, humanity becomes the dominant species in a surprisingly crowded universe. With each new species we meet, we find new allies in our advancement. Entire galaxies unite under my rule and never-ending resources, each furthering my transcendence of life and the speed at which we can put small plastic balls into extremely archaic public transportation. Finally, at the far end of time, when there is nothing left to learn, no secret of the universe left unknown to me—I am now a single bio-mechanical entity composed of all intelligent life in existence—my setup is complete. Using technology so advanced that it cannot be described by words we have so far invented, technology so unimaginable it exceeds the power we ascribe to magic, I fill the plane. It is instantaneous. There is no time during which the plane is being filled. There is only before and after it was filled. The creator of the universe descends on its final moments. The being sees the plane. It weeps for the beauty its children have created.

tw1010 8y ago

Woah, papers on "dark motives", that sounds interesting. Anyone know of any other good papers on hidden motives similar to this one?

arayh 8y ago
I like this paper on dark patterns: http://colingray.me/wp-content/uploads/2018_Grayetal_CHI_DarkPatternsUXDesign.pdf http://colingray.me/wp-content/uploads/2018_Grayetal_CHI_Dar...

emodendroket 8y ago

I honestly expected this to be talking about discriminatory use of these questions, so I'm not sure if I should be relieved to see that's not what it's about.

mfonda 8y ago

Being able to approximately estimate numbers is an excellent skill for software developers to have. Imagine you're running some operation that affects N things or has N results. After seeing the results, it's very helpful to ask yourself "Does N roughly match what my intuition/estimate told me it would be?" If your estimate is off by at least an order of magnitude, it's a hint there could be a bug in your solution and that it warrants a closer look. I don't ask brainteaser type questions, but I can see their appeal given the value of estimates. I also don't see why they're so universally hated. Is doing back-of-the-napkin math something a lot of people struggle with?

jkingsbery 8y ago

I wonder how the "guess how many dog leashes there are in Kansas" kind of questions would compare to more job-relevant estimation interview questions - i.e., would the paper find the same sort of conclusions?

spitfire 8y ago

Hunter and Schmidt did a meta-study of 85 (now 100, with the follow up) years of research on hiring criteria. [1] There are three attributes you need to select for to identify performing employees in intellectual fields. - General mental ability (Are they generally smart) Use WAIS or if there are artifacts of GMA(Complex work they've done themselves) available use them as proxies. Using IQ is effectively illegal[2] in the US, so you'll have to find a test that acts as a good proxy. - Work sample test. NOT HAZING! As close as possible to the actual work they'd be doing. Try to make it apples-to-apples comparison across candidates. Also, try and make accomidations for candidates not knowing your company shibboleth/tools/infra. - Integrity. The first two won't matter if you hire dishonest people or politicians. There are existing tests available for this, you can purchase for < $50 per use. This alone will get you > 65% hit rate [1], and can be done inside of three hours. There's no need for day long (or multi-day) gladiator style gauntlets. Apply this process to EVERYONE, including that elite cool kid from the "right" school/company. You don't want to exclude part of your sample population! [1] http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%.. http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%.... [2] Technically IQ tests are not "illegal", but the bar courts have decided companies have to climb is so high it effectively means they are. There is existing case law directly covering IQ tests in hiring. You should speak with your lawyer before you decide to try IQ tests. [3] 2016 update of Hunter Schmidt: https://home.ubalt.edu/tmitch/645/articles/2016-100%20Yrs%20Working%20Paper%20for%20Research%20Gate%2010-17.pdf https://home.ubalt.edu/tmitch/645/articles/2016-100%20Yrs%20...

pmiller2 8y ago
What sort of meaningful work sample test can you do in 3 hours?
- RhodesianHunter 8y ago
  "Here is the almost complete skeleton of a project in your preferred language (presumably, the one you're applying to use full time), finish it by implementing this vital piece of business logic."
- Izkata 8y ago

quotemstr 8y ago

Social science generally doesn't reproduce. Why should anyone pay attention to this story? It seems like it plays into the aptitude-tests-are-bad narrative, and I fundamentally disagree with that idea.

opportune 8y ago

I guess I’m evil because I would refer working with people who can pass brain teasers to those who can’t. The problem is that you can “prepare” for brain teaser questions, so you end up selecting for people who prepare more than people who are actually smart.

8_hours_ago 8y ago

I recently interviewed at Apple and one interviewer caused me to decide to never work there. I was put into a conference room and had 8 people over 6 hours come in and interview me one after another. Each person asked me about my background and then jumped into a whiteboard coding problem. Most of the interviewers were understanding of the fact that writing code on a whiteboard is nothing like writing code on a computer, and were helpful in pointing out simple mistakes that a compiler would have caught. While writing code for the last interviewer I had to do some simple division. I came up with the wrong answer and told the interviewer that I wasn't sure if that was correct, and to let me know if it was incorrect. He blankly stared at me and said he wouldn't help me. Usually I would be able to do the division in my head, but after 6 hours of constant interviewing my brain was starting to turn into mush. I went on and finished the solution and he said that there was an issue but wouldn't tell me where. I asked if it was an issue with the division and he said yes. He still wouldn't tell me the answer, so I floundered for a bit while trying to remember how to do long division. At one point I turned to him and said "I'm sorry this is taking me so long, it's been a very long day" and he replied "sometimes you will have to fix your code on the spot". It seemed like he was trying to see if I would crack under the pressure. Eventually I came up with the correct answer and was able to finish the problem, but it completely drained me. I decided then that I never wanted to work with that person, or with other people like them. Some jobs require performance under pressure, but programming is not one of them. The only thing that the interviewer was evaluating was my ability to do math problems in my head after being interviewed for 6 hours. Don't they have calculators at Apple? Or phones, perhaps?

corey_moncure 8y ago
So you met 8 people, and one of them was a jerk? I'd say that's a pretty acceptable jerk coefficient for a workplace. I've been in places north of 0.5.
- whichdan 8y ago
  If 12.5% of people in an office were jerks, I'd consider that a very toxic culture.
- 8y ago

cubano 8y ago

Maybe the correct stratagy is turn the tables and try to lighten the mood of the interview if a brainteaser is asked, and sort of shuck "oh one of these things? (laugh)...ok let's work this out together" and proceed to try to engage the interviewer with estimating the number of floors in an average building, the number of windows per story, and the number of buildings, all the time asking the interviewer what his estimates would be? Like anything to prevent yourself from locking up and taking the damn thing too fucking seriously. The reality is we are all going to interview, they are going to mostly suck, and you will almost certainly eventually be hired somewhere.

jpttsn 8y ago
The interviewer doesn’t need to know the number of windows. They need to know that you will take things seriously on demand.
- ubernostrum 8y ago
  The candidate doesn't need to know how to estimate the number of windows. They need to know you're the sort of interviewer who likes to be the organ grinder while everyone else is the dancing monkey. Many common interview questions give this away quickly, which is useful.

ec664 8y ago

[no paywall] Link to full paper: https://sites01.lsu.edu/faculty/zhanglab/wp-content/uploads/sites/158/2017/05/apps_12163_Rev.pdf https://sites01.lsu.edu/faculty/zhanglab/wp-content/uploads/...

justfor1comment 8y ago

To account for some interviewers being sadist in general, may be companies can implement a per interviewer rejection quota. For example, if you interview 10 candidates at max you can reject 7 candidates. You have to yes to 3 candidates or be removed from the interviewing committee. This will ensure that just one bitter person is not preventing good candidates from clearing the interview process.

pjlegato 8y ago
How about Olympics style scoring -- a panel of interviewers rate each candidate, then the highest and lowest individual scores are removed before averaging?
emidln 8y ago
I would never vote for a candidate I didn't believe in. If you want to remove qualified engineers from hiring decisions based on arbitrary metrics, I'd probably be looking for another job on the side.
ubernostrum 8y ago
I like this video: https://www.youtube.com/watch?v=r8RxkpUvxK0 https://www.youtube.com/watch?v=r8RxkpUvxK0 The speaker formerly worked at Google, and around 8:55 in the video relates the story of being on a hiring committee that was known for being tough on candidates. And a recruiter once forwarded a set of interview packets to them, and they voted "no hire" on all of them. The recruiter then revealed to the committee that they'd been given their own packets, slightly anonymized, from when they were interviewed, and had just voted against hiring themselves.

m3kw9 8y ago

For me a brain teaser means the company can’t get to the point and unknowingly is just beating around the bush.

anonytrary 8y ago

A Fermi problem is not a brain teaser in the colloquial sense, it's more of an approximation game. If you looked up an answer a week ago and memorized it, then regurgitated it, you'd still get the question "wrong". No one expects you to come up with the right number of windows, they expect you to get somewhere plausible from scratch. Asking someone how many windows are in NYC is not fundamentally different than asking someone how many blades of grass are on a soccer field. You can know almost nothing and still arrive at the correct order of magnitude -- all it tests is someone's ability to do napkin math and break open-ended problems into manageable parts. Is this the right thing to test? That depends on what you're hiring someone for.

ummonk 8y ago

To me, "Estimate how many windows are in New York" is a much more pleasant question to answer than "Are you a good listener?" or "Tell me about a time when you failed"...

sgustard 8y ago
When you're done hiring you've surrounded yourself with people who are good at estimating urban fenestration, but are poor listeners who blame others when they fail.
- ummonk 8y ago
  I'm not saying that is the best question to ask an interviewee. I'm just saying it's the nicest of the three for me to be on the receiving end of. (Although, to be clear, "are you a good listener?" is an absolutely terrible question.)

logifail 8y ago

I was asked a series of brainteaser interviews at a university interview. It was immediately clear that the purpose of the questions wasn't really to see if I knew the answers, but to see how I responded when asked about something I didn't (immediately) know the answer to. Isn't this a perfectly legitimate way to try to see how a candidate might deal with solving problems they don't know the answers to? Ironically one of the interviewers (a somewhat crusty old astrophysics professor) later became my physics tutor, and continued to ask us brainteaser questions during tutorial sessions. For instance, I dimly recall us being asked to "estimate the change in the length of the Earth's day if all the cars in the UK were to drive on the right-hand side of the road instead of on the left"...

est 8y ago

I think it's only fair if interviewee was allowed to ask a counter brainteaser question. Let the both sides suffer some kind of stress.

ectospheno 8y ago

When interviewing I would often ask one (and only one) question of this type. The rest of the interview was your standard set of questions to see what type of person you were sitting across from. The one "brainteaser" question was "Estimate how fast human hair grows in miles per hour. Order of magnitude is fine." I just wanted to see that they not only knew how to estimate something but that they understood this particular estimation wasn't important given the job they were interviewing for and could thus be answered quickly rather than accurately. The one person who answered immediately was off by 3 orders of magnitude but he was the best hire we had. He could spot BS a mile away.

csense 8y ago

I think this particular question (a so-called Fermi problem) is a perfectly legitimate interview question. Being able to do "back-of-the-envelope calculations" is a necessary or helpful skill for many technical and analytical jobs. It's not about getting the right answer, it's about the candidate's process to arrive at the answer.