23 ms·

Why we don't hire programmers based on puzzles and tricks

• jmduke 14y ago
  Eight minutes, 21 points, zero specifics. I don't understand the 37signals love sometimes.
  • RyanZAG 14y ago
    I think people are up voting this one based off just the message, rather than the content. The message is important, I guess. Plus a lot of 37signals staffer's and friends use HN - they will likely be up voting this based off a Facebook post or similar. ~10 of the 21 points being users affiliated with 37signals is very likely.
    • LockeWatts 14y ago
      Doesn't this violate the HN etiquette?
      • pseut 14y ago
        The discussion's been useful, though.
  • jebblue 14y ago
    They maintain a fresh, honest, pragmatic view of software development and engineering. Those are qualities people can aspire to and identify with.
    • pdeuchler 14y ago
      It seems more like they make a risk-free blog post parroting general HN sentiment every couple of weeks for the karma/publicity. Not that fresh or honest if you ask me, and hardly pragmatic.
      • modoc 14y ago
        I seriously doubt that 37Signals authors blog posts for the purpose of getting karma on HN.
      • xentronium 14y ago
        DHH strongly despises HN[1]. [1] https://twitter.com/dhh/status/289669574427820032 https://twitter.com/dhh/status/289669574427820032
        • papsosouid 14y ago
          He says on twitter. To gain more publicity from HN. Just because he hates that there's some people here who call him on his bullshit, or even if he hated HN for real, that wouldn't mean he doesn't use it for marketing.
          • xentronium 14y ago
            They obviously can't prevent other people from submitting links to their blog, but I don't see them submitting their own content either. http://www.hnsearch.com/search#request/submissions&q=37signals.com&start=0 http://www.hnsearch.com/search#request/submissions&q=37s...
            • papsosouid 14y ago
              If they actually, honestly didn't want their inane spam posted on HN, they could easily achieve that simply by returning a non-response or redirect or error for any requests with a HN referer header. They do not do that because they actually do want to continue getting traffic from the inexplicable portion of HN that insists on posting their crap all the time.
• RyanZAG 14y ago
  I think in addition to the measures he says, if you're hiring for a web position, it's good to make sure they understand how the stuff they're using functions. If they don't have a good grasp of how http works / url parameters / that kind of thing, then they can have some nice looking code which seems to work, but has faulty assumptions that can be security and bug nightmares down the road. Of course, you could always take smart people and train them - but seriously, who does that anymore?
  • UnoriginalGuy 14y ago
    > Of course, you could always take smart people and train them - but seriously, who does that anymore? Great, now I am depressed...
  • hippee-lee 14y ago
    > Of course, you could always take smart people and train them - but seriously, who does that anymore? The companies who get tired of whining that there is no tech talent to hire.
    • RyanZAG 14y ago
      I've never actually seen that happen. This is what happens: 1) Large companies will import H1Bs and/or open development offices in India and similar. 2) Startups will have the founders learn to code (this often ends well), or they will outsource the development work (this often ends badly). 3) Companies will hang outside universities at graduation/trade shows and try to grab promising students (this is mainly restricted to top tier universities) You may have good examples of it happening though?
      • josh2600 14y ago
        We hire folks who have either a CompSci background or "skillz". If you can walk the walk a piece of paper is just signaling. At 2600hz we have folks with Masters Degrees and folks who learned to code the hard way, but it's important to understand that it's easier to start with a foundation and build than it is to start from scratch. That being said, old habits die hard and it can be difficult to unlearn them. Ultimately, referrals are almost always the best source of candidates. The best way to interview is with a conversation. You can learn just about everything you want to know if you ask the right questions.
• thesis 14y ago
  Ok... I was under the impression that 37Signals hired mainly remote workers. Could this account for the lack of a white board? Personally, I use a whiteboard when hiring. It's not the deciding factor... but it's a part of the bigger picture. As part of a small start up I need to know right then and there if they know what I need them to know.
  • dcope 14y ago
    Even though there are plenty of online whiteboarding tools, I'm pretty sure 37signals can afford to fly someone in for an interview. To say the lack of asking whiteboarding questions because they work remotely is absurd.
  • blktiger 14y ago
    I think using a whiteboard to test someone's design knowledge is probably a good idea. I like to draw diagrams on a whiteboard when I'm designing a solution to something. However, using a whiteboard for a CODING exercise is stupid. When was the last time you wrote any significant amount of code on a whiteboard? Give someone a laptop if you want to see their coding skills. Even better, have them pair-program with one of your developers.
• dmbaggett 14y ago
  At ITA Software, we found in-person quizzing to be a lot less useful an indicator than looking at puzzle (or other) code someone had written "offline". We also found puzzles to be a useful talent magnet. If you're Google (or perhaps 37Signals) you don't need a talent magnet. If you're a small company nobody's heard of, one can be quite helpful.
  • hexagonc 14y ago
    I remember actually working on some of the puzzles that were posted by ITA in the Boston MBTA subway. I didn't submit them (since I wasn't interested in working at ITA) but they introduced me to interesting problems. For example, there was a problem related to generating rebusses* whose solution contributed to the solution to another problem I was working on. *http://en.wikipedia.org/wiki/Rebus http://en.wikipedia.org/wiki/Rebus
    • dmbaggett 14y ago
      That one was mine (in its original conception, at least). :)
  • mapleoin 14y ago
    I think it's the opposite. If the first thing you confront me with in your interview process is a puzzle and I don't know anything about your IT department, I'm going to assume that all your managers are pointy-haired bosses.
    • dmbaggett 14y ago
      Empirical evidence from over 10 years suggests you're an outlier.
      • danielweber 14y ago
        This is just signalling. "Do X and I will think you are an idiot!" is his way of saying "please don't do X."
• dschiptsov 14y ago
  Because no one will hire a writer based on his knowledge of long words.) Excellence in the grammar is also irrelevant.
  • petercooper 14y ago
    And a doctor doesn't necessarily have to be fit and healthy.
    • dschiptsov 14y ago
      That's different.))
  • fnordfnordfnord 14y ago
    Reminds me of this supposed exchange between William Faulkner and Ernest Hemingway. “He has never been known to use a word that might send a reader to the dictionary.” –William Faulkner (about Ernest Hemingway) "Poor Faulkner. Does he really think big emotions come from big words?" —Ernest Hemingway (about William Faulkner) >>Excellence in the grammar is also irrelevant. It took me a long time to grasp the fact that there are very many intelligent and skilled people who have dismal grammar and spelling skills. It still troubles my mind when I encounter examples of it.
• zdw 14y ago
  The best questions to ask are the ones that pertain to actual work. Think of a problem you had before and solved or are attempting to solve. Ask the person being interviewed how they'd handle the problem. Do they have good process, intuition, and problem solving skills? Are they clear about communicating their intentions and explaining their decisions? This isn't a trick - it's what the person is going to be doing there, in and out, every day.
  • pwny 14y ago
    Completely agree. I recently had an interview where I was asked to solve a job scheduling type problem. So I see the potential graph, remember topological sort and bingo! Right after that the interviewer told me they have a lot of code that manages assets that have priorities between each other and that this is the kind of problem they solve regularly. It was a good question: it wasn't too hard as to take hours to solve, it was a decent test of how I was able to choose an appropriate representation for the data and it applied to a real world problem they solve.
• ambiate 14y ago
  Basically, most of the questions I see asked in interviews are contained in Levitin's algorithms book. The greatest potential for someone to excel at those trick interviews is to be fresh out of college. I walked out of my first interviews, because 'I do not like programming games.' The first was a 4d matrix puzzle with more mathematical/theory roots than programming/logic. My third interview started with the above quote and I was hired on the spot -- I believe they took it as a rebellion of every fresh out of college kid wanting to program video games.
• paf31 14y ago
  Shameless plug: I've been working on this : http://initialround.com http://initialround.com to allow applicants to complete an interview at home and in a browser, to eliminate the pressure of the "quizzing cage". I'd be interested to hear people's take on this approach.
  • wavefunction 14y ago
    My current position did something similar with an existing FiddleJS they had. When it was time for the initial screening, we spoke on the phone while I forked the Fiddle and "whiteboarded" in real-time. I thought it was great and I urge you to pursue your project further.
    • paf31 14y ago
      Thanks! I've done similar things with online text editors, and wanted the ability to compare applicants side-by-side. I was also thinking this might be useful for recruiters who don't necessarily know the code, but just want to pick questions from a preset list to get started.
• potatolicious 14y ago
  Damn right. Even worse than whiteboarding random algorithms are logic puzzles. I can sort of see how implementing my own hash table fits into a dev job, but I why manhole covers are round, or light bulbs in a room has absolutely nothing to do with anything relevant in our field. People who use logic puzzles as proxies for anything in an interview have my everlasting scorn.
  • smoyer 14y ago
    Logic puzzles are definitely useful ... how else can the interviewer feel superior (after all, he/she already knows the answer and has probably forgotten how they came by that answer).
    • rdtsc 14y ago
      If I was given a logic puzzle I would ask them "so how many times in the last year this company had to solve this particular puzzle here?".
      • mark-r 14y ago
        And you would have gotten an instant fail on the "cultural fit" criteria.
        • rdtsc 14y ago
          Good!
          • mark-r 14y ago
            Just as a logic puzzle might be a bad predictor of how you'll perform on the job, it's probably also a bad predictor of the quality of the company doing the asking.
  • tiziano88 14y ago
    I think that question about why manhole covers are round is more of a urban legend than anything else. Or it might be a question asked to someone applying for a non-technical role, to see how they reason about the issue (though it is most definitely not the best question to achieve so).
• csmatt 14y ago
  I remember when I was prepping for my Microsoft interview. There was so much lore online as to the types of questions they'd ask and how to prepare. I even found 'How Would You Move Mount Fuji?' in the library and started reading through it. It was an absolute waste of time as I, fortunately, learned weeks before the interview. As an example, one of the questions I was asked was 'how would you write the C function strcmp() without using any libraries?' My gripe these days is that places I've interviewed with now that I have a few years of experience are still concerned about big-O notation and a lot of the more academic questions. I feel like I shouldn't have to go back to my college notes to prep for an interview for a job that I'd really be using my current experience for. I think it comes down to other developers not spending time really thinking about the quality of their interview questions. They just do a search for 'programming interview questions' and call it a day. My most recent interview was with Amazon. I was asked the dumb big-O runtime stuff as well as how I'd count all of the stars in the universe. It pissed me off, but I did like the question I was asked about how I'd design an elevator system and then asked how my solution would scale. That's not too far off from what I'd be doing, albeit in an abstract form.
  • rdtsc 14y ago
    > and a lot of the more academic questions It is on purpose. A <bigcompany> recruiter confessed. He said they are specifically targeting recent grads. So plenty of questions are of that type. I guess they want to indoctrinate and take advantage of them while they are "fresh". > did like the question I was asked about how I'd design an elevator system and then asked how my solution would scale. That's not too far off from what I'd be doing, albeit in an abstract form. That would piss me off though too. "So does Amazon write algorithms for its elevators?". Why not just simplify a problem they solve. "So you have key value storage and you want to make it distributed". Why even talk about elevators?
    • saosebastiao 14y ago
      Actually, yes...they do. I work in a building with an elevator control system designed by an internal engineer. More importantly, the elevator control system uses mathematical expertise that is normally used solving variants of the multi-commodity flow problem, used extensively in the Supply Chain.
• elliottcarlson 14y ago
  I've been on both sides of the hiring process - and most recently was going through the interview process again as I sought out my next opportunity. One thing that stood out to me was the way companies would approach the white boarding or coding challenges. While I agree with the articles issue with coding up a specific question (especially when it is yet another fizz-buzz - even if it has a twist to it) - the puzzle style interview can provide some kind of insight in to the thought process behind someones work. One company in particular really stood out by having a great white boarding exercise that not only was entertaining, but offered a decent amount of a challenge that it really made them a frontrunner in my final decision making process. A good coding interview can be beneficial not only to the hiring company, but can also give decent insight in to how the company works for the interviewee - and that is something that should be equally important if you are trying to attract talent.
  • notimetorelax 14y ago
    Do you think you can elaborate a little more on the exercise that you find interesting?
    • elliottcarlson 14y ago
      On one hand I would love to - and on the other hand I don't want to give this companies interview question away out in the open. If you want, you can email me (email is in my profile) and I would be more than happy to share it directly!
      • notimetorelax 14y ago
        Fair enough. It's OK I asked only out of general curiosity and to compare my own experience with yours.
• bromang 14y ago
  is there any well known data/studies on selection methods for programmers? I can understand the reasons why not to use algo problems as a filter, but I would have thought simpler forms of intelligence testing would still be quite useful.
  • fnordfnordfnord 14y ago
    I think Token has the longest list of that sort of thing. I'm sure he'll be along shortly to post it ITT. Dig through his comments here if you can't wait. http://news.ycombinator.com/threads?id=tokenadult http://news.ycombinator.com/threads?id=tokenadult
• SGemployee 14y ago
  Employer told me "We do hardcore stuff!" come on really hardcore?! I would really love to post their apps here but I'm not that kind of person.
  • Dirlewanger 14y ago
    Brogrammers, man. They are not a myth.
• rdtsc 14y ago
  Exactly. I never ask those questions. I ask practical problems that we solve every day here. Some are simplified of course. Also I don't try to be confrontational, I take the "let's solve this together". That often puts them at ease and makes for a better experience and a more productive interview. Now it is still an interview and the selection process could be brutal but this way I get the most _relevant_ knowledge and information about the applicant in the shortest amount of time.
  • elliottcarlson 14y ago
    The "lets solve this together" approach is always great - I've had people simply give up and not ask any further questions - those are generally "pass" situations. I rather see someone be genuinely interested in how to solve the question - the more questions they ask, the more I will be willing to help them, and the better they will actually do in the interview.
    • LockeWatts 14y ago
      The problem with this though is the meta-strategy of interviewing. Unless your interviewer tells you up front whether or not it's okay for you to ask an unlimited number of questions, you're basically guessing at it. I know several companies that interview with a "hint" system, where every time you ask a question, they just give you a hint to the problem, generally regardless of the question asked, and then mark it down. Too many hints and you're passed over.
      • elliottcarlson 14y ago
        Interesting - while I never said it was OK to ask unlimited questions I have generally gauged how someone is doing and then let them know it's OK to ask questions or need hints. Some people have accepted the offer, and some people have simply given up - which just astonishes me.
      • rdtsc 14y ago
        How many questions they ask, how often and how relevant those questions are all taken into account when grading communication skills. Some are just way to shy and never ask question. That's a slight negative. I am shy too, but I need to get the job done I'll go find out who to talk to, ask question, figure out specs, see who's done what before and so on. That is an important trait. I'll give out hints as well, it is not too structured. I don't immediately subtract some point or anything. In fact I might hold it against them if they don't ask questions.
• SeanDav 14y ago
  All great and well spending 20-40 hours trying someone out to see if they are a good fit, but this is only practical once you are down to a very short list of final candidates. Nothing is said about how you actually get to this final list and that of course is where the real challenge lies.
  • bcoates 14y ago
    Whatever happened to picking at random? It's optimal unless you know some other method of narrowing down the list isn't pathological.
• btipling 14y ago
  Eh, being able to understand stacks and queues, big-O, etc and problem solving are important things, but otherwise agree that software construction skills are important as well as understanding tradeoffs bottlenecks, networking etc.
• fnordfnordfnord 14y ago
  It's too easy to prepare for the common ones. I have a list of them which I share with my students before they graduate. Sometimes we work through them for fun.
• buro9 14y ago
  The question I always like to ask and reserve a good chunk of time for is along the lines of "What non-technical hobby or interest do you have?" and then "If you had all the technical resources you could dream of, how would you now bring something new to your hobby?" and once we've gone through that the real question is: "Given that when you're duck hunting the key is to shoot ahead of where the duck is, and that a lot of the suggestions have been the next iteration applied to the hobby/pastime... what's the third or fourth iteration?". Basically I want to see the excitement and passion about technology, their pastime, and how they think, how they identify problems, solve them in numerous ways, go beyond just today's solution. And I get wrapped up in it too, I also get excited by some of the stuff that comes up. With a great candidate... the interview ends with us both inspired and a thousand new thoughts floating around the room. The vast majority of candidates cannot leap beyond the first iteration, and some struggle to even see how anything within their hobby could ever change.
  • danielweber 14y ago
    And when asked about your hobby, don't say "shoot hoops." (referring to http://news.ycombinator.com/item?id=5255227 http://news.ycombinator.com/item?id=5255227 )
  • papsosouid 14y ago
    >What non-technical hobby or interest do you have? That seems like a poor choice. Are you trying to eliminate people who don't have non-technical hobbies for some reason?
    • buro9 14y ago
      It's fairly irrelevant, the last part is the important part... very rarely I do find someone who stubbornly declare that they have no other interests at all, but then I just propose for them some pastime or other that may fascinate them.
      • papsosouid 14y ago
        >very rarely I do find someone who stubbornly declare that they have no other interests at all Of course, but your wording rules out a large number of likely interests as they are technical.
• mnemosyne51 14y ago
  I hope someone from Goldman Sucks reads this.
• cletus 14y ago
  Sigh. This is the issue that will just never die. Let's just summarize the points: - Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here); - Side projects, even open source projects, can be of questionable "real world" value; - I've personally encountered many "programmers" who can talk a good game but can't code a for loop; - tests like the FizzBuzz test [1] are, in my experience, remarkably effective as an early negative filter. This is an important point. If someone blows you away at FizzBuzz, it doesn't mean they're an awesome engineer. But if they can't do it, it almost certainly means they aren't. The idea here is to spend the most time with candidates who might potentially work out and wasting as little time as possible on those that probably won't; - the problem with these kinds of whiteboard coding problems is that the tendency is for interviewers to think the problem needs to be "hard". It doesn't. In making it too hard (IMHO) you risk destroying the value of your filter; - pop-quizzes of obscure language features, the kind that might appear in certification exams, are a waste of time. I have no argument with that; - whiteboarding code by itself is not a great filter. It should be used in conjunction with a multi-faceted interviewing approach that involves testing fundamentals, the ability to construct a relatively simple algorithm, the issues of working on a team and on a production code base and systems design. - the problem with simply talking about "real world" code, as the author suggests, is you're no longer finding a good engineer, you're finding someone you like, someone who thinks like you. This falls under the umbrella of cultural fit, which is of course important, but don't mistake that for engineering skill. - I think we can all agree that "logic" puzzles like "how would you move Mount Fuji?" or "if you shrunk to 1cm in size and dropped in a blender, what would you do?" are stupid. - testing "back of the envelope" estimation however can be useful. I mean things like "how much storage is required to store satellite images for Google Maps?" The idea isn't to get an accurate answer. It's to see what assumptions the candidate states and, based ont hose assumptions, to come up with a reasonable ballpark number. The problem here is that there are many engineers who can't comprehend the possibility that there is someone being paid to be a programmer who can't code. But I assure you this is the case. It's shocking but true. Simple coding tests largely filter these people out so if you're offended by such simple tests, just do it and move on. I assure you there's a reason why they exist. One final prediction: There's some guy here on HN who always posts the exact same huge comment on any hiring thread. I'm sure it'll pop up any moment now. EDIT: let me add a point about trial periods and take home assignments. Both of these are guaranteed recipes for mediocrity. Truly outstanding candidates need to justify the time investment for either option and very few companies have the kind of gravitas that would justify it. Anything written without supervision will be of questionable provenance at best. As for whether or not someone will work out in your organization, bringing them in for a day is (IMHO) of questionable value. Many engineers are introverts. I include myself in this. It's incredibly awkward as is to be in a new company or even a new team in the same company. I question the value of any such assessment over what you learn in 1-4 hours of interviewing. [1]: http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html http://www.codinghorror.com/blog/2007/02/why-cant-programmer...
• flexie 14y ago
  These puzzles and IQ tests are as pervasive as they are irrelevant. Nowadays, you even need to pass one to get some government jobs. Here's the one to pass to get a job in the EU agencies: https://europa.eu/epso/application/passport/quiz.cfm?lang=EN&comp_id=1&quizid=10&f_sub=+OK https://europa.eu/epso/application/passport/quiz.cfm?lang=EN...
• blparker 14y ago
  I consider myself a proficient developer, however, in situations where I'm required to solve an obscure puzzle while someone is staring at me, generally makes me uncomfortable...no matter how easy the question. I interviewed for an internship position at Microsoft, where I was required to write some bizarre code on graph paper while the interviewer literally stood over my shoulder. Horribly uncomfortable, I barely made any progress, and was subsequently denied a position (obviously). After the interview, I went home and solved the question in a matter of minutes, wrote the code, optimized, and fully tested it. While I choked during the interview, I saw fellow classmates fly off to Redmond...classmates that I knew weren't better developers than I was, simply because I had worked with many of them on projects several times. That's when I knew this whole puzzle interview stuff was nonsense.
  • jdmichal 14y ago
    I have a similar story with Microsoft when I interviewed with them my senior year also. I feel like I nailed everything except for the puzzles, and also got denied. My favorite one I "failed" on was: Given a node of a linked list, delete that node. The expected answer: Copy the next node's contents into the given node. Then delete the next node. Just simply never occurred to me to use a memory copy in a linked list. And this is despite the fact that I could immediately reason about the solution he gave me and point out a flaw in it (deleting the last node is impossible).
    • boomzilla 14y ago
      but that is not a puzzle, it's real code if you don't access to a collection library
      • jdmichal 14y ago
        No, it's a puzzle for two reasons: 1. It's not a normal operation for a linked list. A normal linked list will expose operations to deal with items, not nodes. 2. There was an expected answer, and that answer was something other than the normal linked list delete operation of "traverse the list to find the node containing the item, while also keeping a trailing pointer to the previous item so that you can remove that item's node." Even if you are exposing nodes, and you do implement the given operation, then TheCoelacanth is absolutely correct that that operation just invalidated all references to the node after the one you thought you had just deleted, but not any references to the node you thought you had deleted. (Though those references are now pointing to a different data instance.)
• pdeuchler 14y ago
  Interviewers ask puzzles and give you tricks because they want to see your problem solving process. Obviously, anyone who hires simply off a litmus test of such questions is not doing their job, but that doesn't preclude the use of such brain teasers/problems at all. Asking a candidate a difficult brainteaser that may be even close to impossible can be extremely revealing. There isn't a canned response, you get to see them work under pressure, you experience live problem solving and you can understand how they move through their work better. All of these factors are just as important as competency when hiring. A great programmer doesn't "fizzle" in front of a difficult interview question. He/she may or may not get the correct answer, but as long as they have the requisite problem solving skills and communicate with their interviewer well they will come out fine. I'd say that showing how you fail can be quite beneficial when trying to get a job, especially if you happen to fail gracefully :) It seems like 37signals is just echoing platitudes to gain karma/publicity at the expense of cheapening the discussion.
  • herdcall 14y ago
    All good, but the success of this method also depends on how well the interviewer interprets/evaluates the interview. Unfortunately, the danger here is that inexperienced/immature interviewers can easily discard good candidates this way. This is similar to the skill required to grade a written test for partially correct answers vs. right/wrong ones. Hence the criticism on wide-spread usage.
  • gizzlon 14y ago
    People can be nervous on job interview and therefore "shut down" and fail simple tests. This does not reveal a persons real problem solving skills. I don't consider myself especially nervous in job interviews and when taking tests and exams. But I still had a much harder time solving simple programming tasks than I would "back at my desk" with no-one watching. YMMW. It would suck to want a job so bad you completely freeze up on simple interviewing tests :(
    • pwny 14y ago
      On the other hand, if I'm hiring someone for a high pressure job I don't want them to fail when under pressure. I understand your point but I don't really buy it. I train lifeguards as a hobby/side-job (I'm in uni right now) and pressure is the number 1 reason they give us for failing their final pratical test. I can't give a kid a permit to work as a lifeguard if the pressure of an exam makes him screw up because the real life pressure of saving somebody's life is even greater. The exact same applies here for all jobs where you expect the engineer to work in stressful situations.
• andrew_wc_brown 14y ago
  I won't present a résumé, a github link, a linked in profile, or do nonsense tests. I always send a full web-application I built for code review, do a technical demonstration and provide sample work on their codebase. Any other route is a waste of time.
• deleted 14y ago
  [deleted]
• misiti3780 14y ago
  I have heard google is a big violator of this?
• ctdonath 14y ago
  [I] got asked how to do something in JavaScript on a white board. The specifics are vague, but it’s crystal clear how stupid it made me feel and how little it had to do with the actual job. So did I. (Well, it was C then, but same idea.) The prospect of scrawling code on an inapplicable medium while people stared at every move is, of course, irritating. Instead of going to the whiteboard, I pulled out my notebook computer, fired up a code editor, and told the several guys present to continue the interview _while_ I worked on the challenge. Time was saved by multitasking, I did the work in a sensible normal manner, everyone was comfortable with the situation, and they were happy with the resulting code. I got the job. And turned it down.
• GhotiFish 14y ago
  If I can have a word on FizzBuzz? There are a few programmers that have a proclivity towards "clean, perfect" solutions, but FizzBuzz is ugly. Making FizzBuzz work at all is just stupidly simple; but the moment you ask yourself about the little details, you notice that you either have to make an if tree, a bunch of if-elseif's, or some other hinky thing that ruins the "purity" of this simple little program. Alot of people take the perspective that if you go if(i % 15) print "fizzbuzz"; else if(i % 5) print "fizz"; //or was it buzz? else if(i % 3) print "buzz"; //or was it fizz? else print i; then you've found an optimal solution and you can wash your hands of it, until you think to yourself: "but an if tree would use less comparisons! Should I be readable? Or take the efficient route? But if I'm thinking about efficiency. wouldn't a lookup be faster and simpler?" I suspect when presenting a problem like this, the interviewer filters out two kinds of people: the people who "can't code", and the people who are obsessive.
  • papsosouid 14y ago
    The entire point is "is this person capable of the most basic fundamentals of programming (conditionals, looping). That's it. We don't filter out obsessive people with it, they demonstrate that they understand conditionals and looping, what's the problem?
    • GhotiFish 14y ago
      I think FizzBuzz does filter obsessive people. Someone who is obsessive wouldn't like their off the cuff solutions to FizzBuzz. They would think of a solution and stop themselves from writing anything at all. How would you tell the difference between someone who doesn't want to write their first three solutions, to someone who can't think of even one? Microsoft's "implement strcmp without the use of libraries" doesn't have this property. strcmp has an easy and clean solution.
      • papsosouid 14y ago
        >I think FizzBuzz does filter obsessive people. FizzBuzz doesn't filter anyone, the people using it do. It is not difficult to explain the purpose of the exercise, and that easily resolves the problem for people looking for "clean" solutions.
• chadhietala 14y ago
  As someone that just went through a whole slew of interviews I would say I agree with this to some extent. I'm a frontend developer and while CS principals are important, asking questions that are not practical to everyday development, are counter-productive and do not give you a view into how that person tackles a real problem. Things like "write a function that performs merge sort", are bullshit because you would never do this in real life, largely because languages have sort methods that are already extremely performant. Plus a lot of languages will already be using some variant of merge sort and exposing it as a "sort" method. If you are given a problem like "create a tabbed news component" or you're asked to do a small project (nothing longer than an hour), I think you can still get a good understanding of how people solve problems. This is less stressful, less "gotcha" mentality, and a fair approach.
  • mark-r 14y ago
    Merge sort is a bad example - it can actually be useful in real life, because the library sort functions are limited to what will fit in memory while merge sort is not. It's also simple enough to keep in your head without needing to look it up. I've had to code it up on the job more than once.
• tokenadult 14y ago
  As Cletus wrote in his detailed comment, "this is the issue that will just never die." We discuss company hiring procedures here on Hacker News over and over and over, because most of us have applied for a job at least once in our life, and those of us who are building up a start-up have to think about whom we would hire. The point in the submitted blog post by 37 Signals, "The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written, talking through bigger picture issues, and, if all that is swell, trying them out for size," basically says that you should do a work-sample test to hire a programmer. And that's what research says. The more a hiring manager understands what a worker will do on the job, and the better the manager appreciates what a new hire may grow into doing after a few years on the job, the better the manager can devise a work-sample test to identify a good worker. There is a research base for this. Work-sample tests aren't perfect hiring procedures either--nothing is foolproof, and every hiring procedure goes wrong with both "false positives" and "false negatives"--but you can improve your odds of hiring a good worker by using a work-sample test routinely whenever you are hiring. As a job applicant, you can select better rather than worse bosses and co-workers by aiming most of your applications at companies that explicitly use work-sample tests as part of the hiring process. With the help of several Hacker News participants, I have written a FAQ about company hiring procedures, revised over a few months of edits to fit the typical recurrent HN discussion of this issue. See http://news.ycombinator.com/item?id=5227923 http://news.ycombinator.com/item?id=5227923 for the latest posting of that, with full references to the research literature and legal cases about hiring workers in today's world. Feel free to contact me through my HN profile http://news.ycombinator.com/user?id=tokenadult http://news.ycombinator.com/user?id=tokenadult if you have suggestions for improving that FAQ before I post it to my personal website. P.S. Even before I saw the prediction at the end of Cletus's comment, I planned to make the prediction untrue. EDIT TO RESPOND TO FIRST REPLY ABOUT PUZZLE QUESTIONS: Yes, if a company in the United States insists on using puzzle questions as a hiring procedure, and justifies using those puzzle questions by saying that they want to see which applicants are "good problem-solvers," or "able to think on their feet," a rejected job applicant just might be able to subject the company to a very expensive lawsuit based on employment discrimination, unless the company has prepared beforehand a validation study showing that those puzzle questions have a bona-fide relationship to work requirements. I would not advise a company to take that risk, especially when the legally safer alternative of doing a straight-up work-sample test is available. The law is different in other countries, and as a reply in this thread points out, in the EU it is generally legal to use straight-up IQ-type tests in hiring processes, although those are underused by private companies in Europe, according to the sources in my FAQ post. ANOTHER EDIT, TO LINK TO AN UNBELIEVABLE ANECDOTE ABOUT HIRING: In August 2012, I heard a story from a hiring manager of programmers about the hiring procedure he uses as an initial screen for applicants who have degrees in computer science: "Write a loop that displays the numbers 1 to 100." That sounds awfully easy, even to me, but he says that the great majority of his applicants with accredited CS degrees fail that screening test. My earlier telling of the full anecdote http://news.ycombinator.com/item?id=4603414 http://news.ycombinator.com/item?id=4603414 and his http://news.ycombinator.com/item?id=4919749 http://news.ycombinator.com/item?id=4919749 seem pretty nearly unbelievable in what they imply about how clueless many CS grads are, and yet I think the anecdote is a true description of reality. Cletus too mentions in this thread, with considerable agreement from reply comments, that many people hired as programmers cannot actually program.
• louthy 14y ago
  I do tests for prospective candidates, but my approach is slightly different. After the interview I email them a small unfinished program. I then give them a set of instructions on what I would like them to do to finish it off, and they can then do it at home on their own dev setup; with the internet and any other resource they would normally use for development. I give a coding style guide, and in the areas where some specific technique is required I make that explicit. The app has a few bugs in it, and some of the techniques asked for are slightly leftfield. I instruct them that if at any point they have any questions then feel free to email me - and they won't be marked down for asking questions. In fact I mark candidates up if they ask questions. The task shouldn't take more than 30-45 mins for a decent coder. I find this to be a really valuable approach to seeing how a coder works. It shows how they work with others' code, it shows they can follow instruction, it shows they can communicate when unsure rather than hiding away and it gives a good guide to their style of coding. It's a far less confrontational approach, and gives the candidate the best chance of showing what they can do. In the interview I tend to be much more interested in the person, because I'd rather have a coder who is slightly less capable that I can train, than a coder who is amazing at everything but won't fit in. I'll still discuss past projects and dig deeper where necessary, but I like to find out how they influenced the outcomes of projects rather than get too deep on algorithms and the like. So far it's worked very well :)
• rayiner 14y ago
  I used to give interview candidates a little background lecture on our problem domain then ask them for input on whatever I was working on that week. E.g. If you were trying to parallelize this code, how would you structure it? I never asked fizzbuzz style coding questions, because I think they're stupid and I probably couldn't code a linked list on a white board (I rarely code on whiteboards you see).
• kamaal 14y ago
  This is something to think about. The interview process is so badly gamed that these days its kind of a ceremony. In a recent interview, I gave a candidate a practical problem. He was allowed to use the sed man page, and then I gave him a file and search/replace problem. Nothing much! The candidate fails! Instead he asks me if there is going to be a algorithm/data structure interview. I told him, I don't know but as far as I was concerned I was only going to test him on practical everyday problems. It was almost like I had committed a sin. He was like, every company has those standard one's so why was I doing like a practical test. These days all candidates do is go through most common puzzles/algorithms/data structure stuff. Its actually quite easy to get access to those. Just spend some time on interview forums and you can practically game any interview, in any company any where around the world today. My approach is straight simple. Throw a manual/documentation at the person. Give him a problem common enough to test a practical everyday situation. Check if he can pass the the test of reading the documentation and figuring out the solution to the problem. Or give him/her their favorite IDE/text editor + documentation + problem and let them solve it. If they can do it, or even get the approach right they pass the test. Else if all they are going to do is vomit some mathematical theory from college text books, which isn't even remotely relevant to our everyday work- I'm hardly interested in hiring such candidates.
  • mamcx 14y ago
    I do something like that. My first test is ask to reverse a string (not using the reverse function!) in any language and/or pseudocode. Is incredible how badly the test goes with the people! I interview for myself or others more than 100 people (of any background, including university, tech schols...) and I think only 10/12 people do it correctly - barely- To make it more fair, I always let them alone. And give internet and a computer, and remember them they can use internet and search. Not even with that amount of help the thing work well (I truly have find someone smart enough to copy-paste something and return to me in 1 minute or less). Then i do the puzzle question, but know I retire it some years ago and some more question about real stuff...
    • biscarch 14y ago
      This is great. My first thought was "perfect, Haskell!" reverse "whatever" "No reverse function? but we can use the internet? The Prelude source probably has the best implementation of a reverse." foldl (flip (:)) [] "whatever"
• huhsamovar 14y ago
  No need to read. The author of this post has clearly missed the point.
  • tanel 14y ago
    word, to be honest I don't get it why it's on hn.
• Osiris 14y ago
  I'm a self-taught programmer. These stories about programming puzzles in interviews always make me nervous to look for a job. My first job wasn't a programming job, but I did a lot of programming because the need arose and I was the only one capable of doing it. I always felt too inexperienced to look for a real programming job. A friend pushed me to interview for the position I'm in right now (full time web developer). They didn't make me do any whiteboarding or puzzles. What I did do is give them a code sample and we discussed how what I could have done better and why. I got the job and I'm doing quite well, despite my lack of a CS background. Since I don't have a formal CS background, and most of my development experience is web development, I have basically no experience dealing with complex algorithms, but I've found I can solve a hard problem with enough effort (Google, co-workers, etc). For web developers, wouldn't more appropriate questions relate to "please design a database for a website that needs to track the following", or "What would your process be for designing and implementing a REST API that would...". These would allow the developer to show you their thought process of working through a large problem without there being a necessarily right or wrong answer.
• eranki 14y ago
  Why do we always feel the need to upvote posts like these as if we don't discuss them every 2 weeks? Next up in the topic cycle: Discrimination in tech. Why working more than 40 hours a week is killing your productivity.
• army 14y ago
  I think the skills you need to throw together a nice RoR app are different from other positions. I wouldn't want on critical infrastructure code alongside someone who can't reason through the performance implications of their code on the fly. You can waste a hell of a lot of time diagnosing and fixing performance problems because someone built an entire module around an inappropriate data structure.
• lifeisstillgood 14y ago
  About a year ago I travelled to the south west (of UK) for a job - A full day of interviews, product ideas, I nailed the whole lot. Apart from the obligatory coding test. I have 15 years in this game and I have coded OSS or commercial code every day or week in Python, the language of the test, since 96. It was a codility.com test - I had not actually been expecting it but, kerpow. I rewrote my own abs() because I forgot such a thing existed, the problem itself is still blanked from my brain - and every passing minute it got worse. After an hour and a half, the interviewer took pity on me and drove me to the station. I actually sat on the train took the codility demo test. 100%, 3 minutes. Signed up, took more. I could do them. Just impossible to know what went wrong but it went badly wrong. Ultimately it worked out well. I would have had to live weekdays down there and my marriage probably would not have survived it. Now I work 15 minutes walk away and give my kids breakfast each day after we sit on the sofa and watch Mister Maker. For me that test was a wake up call. Firstly, I run my own business - being at the mercy of one boss is rubbish. Secondly, these tests are rubbish for deciding who to hire - but they are OK for programmers as a form of continuing education. Thirdly, the exercises at the end of each chapter of SICP are much much better form of education. Forthly, I was once asked how a large corporate IT shop should handle a reduction in workforce. I said march everyone into a room and those who cant code FizzBuzz get a pink slip Even given my experiences above - I still think that is the best option. Sometimes a guy just dies on that day. Its unlucky. But sometimes it works out for the best. edit: clarity
  • mmorett 14y ago
    I never heard of codility so I went to take the demo test. I couldn't pass that in English, much less Java. Lol. That's not a programming test. That's a math test. That is the perfect example of what not to do.
• KeepTalking 14y ago
  Coming up with good questions to ask in an interview is an effort that the hiring team needs to invest in. This involves a top down reflection of the job along with a vision for the job. It requires to ask him/her self before hand "What sort of tasks is this hire going to be doing" , " What language skills does he need" , " Does he need to reinvent a sorting algorithm to get this job " . Well this rarely happens , to start of folks (generally) tasked with the hiring process are not the right "vision" folks. They are highly task oriented individuals aka MS Project specialists. A real test would have to encompass among many things. - Do they are ask the right questions ? - Are they capable of picking up new languages, ideas , technologies while defending or questioning changes in an articulate yet respectable demeanor. - how they handle things when they get to a wall.
• some_googler 14y ago
  When I came to Google for my onsite interview, in the group of people scheduled for that day there was this guy with an alpha-nerd attitude including a Rubik cube that he kept playing effortlessly -- he could completely scramble the thing and then solve it in seconds almost without looking at it. I was never able to solve more than two faces of Rubik in my life :) fortunately no stupid puzzles at all in the interviews (at least not in mine), I got in. The Rubik guy, never saw again. FWIW.
• fiblye 14y ago
  I was invited to a company for a game development position. One part of the interview consisted of a fairly challenging mathematical/algorithmic problem. It wasn't impossible, but it's certainly not something I can do in ten minutes at a white board with some people taking notes about my every movement. The other part was being shown intentionally messy and broken code. Not code that'd fail to compile (although it sure looked like it would), but something thrown together in a completely insane manner that I couldn't possibly make sense of in 15 minutes. Much of it boiled down to, "What would happen if you call this function and use this variable before you actually define them and then do x, y and z?", and the way this language did it was quite different from other languages. I mean, yes, I did learn a lot about the language from that interview, but I'd really hope that's not the kind of code they'd be throwing at me or expecting me to write at their company.
  • jes5199 14y ago
    Plenty of the gigs I've had have been mostly trying to unscramble really confusing code that mostly-works. If my codebase has a bunch of strange crap in it that I need fixed, I certainly will bring examples of it to the interview.
• aosmith 14y ago
  Couldn't agree more. Every time I've agreed to a "programming test" I find myself wonder why I agreed to it in the first place.
• jwwest 14y ago
  The best interview I ever had, I was given a problem to solve with a system, a whiteboard and 20 minutes. I sketched out a quick ERD, and explained the high level architecture of the system. No code was written -- it was assumed that I could implement said system if I could design it. The worst job interview grilled me on low level algorithms (I'm not a CS grad) and wanted me to sketch out a quick sort. Of course, I can describe a quick sort now as a result of said interview, but since it was all C# I doubt we were going to have to implement our own sort algorithms. Another terrible interviewer asked me about projects I did in college. I finished grad school 3 years ago, and have 8 years professional experience...
  • rdtsc 14y ago
    Exactly. Same here. Now I give interviews, and I do the former not the later. Make sure you do the same if you get to interview others. This puzzle and brainteaser shit needs to stop.
    • mmorett 14y ago
      A. Friggin. Men. I just wish they'd make it easier for us to weed ourselves out. Just come straight out and ask for CS grads only. None of that "or equivalent experience" nonsense. I also don't have a CS degree. I have the bastard stepchild degree (CIS). :-) Just ask for CS ONLY and we can easily avoid you and make this easier on all of us.
  • sssilver 14y ago
    A programmer who is unable to program a Quicksort implementation is like a programmer who can't program anything relatively complex.
• bramcohen 14y ago
  Over time I've made interview challenge problems progressively easier, as I've found that the trickier ones don't give any more information than the easier ones. My challenge now basically amounts to 'read this doubly nested for loop and tell me what it does', equivalent in difficulty to fizzbuzz. If you can't solve that, even in the pressure of an interview, then I'm sorry, but you have no business working as a programmer, and I'm offended that you even applied. And yes, people flunk it all the time.
• ianstallings 14y ago
  Well the good news is this problem should be solved soon since we spend so much time focused on it. I don't care for puzzles. Just conversations. I skim resumes looking for glimpses into personality more than looking for skill sets. I have a guy that went to Yale on my team. I didn't even know that until he mentioned it the other day, yet it was on his resume. I missed it because I don't care where you went. I don't care how fast you can solve a puzzle. It's put up or shut up and show me what you've built when you come here. If you can do that in an interview there's a great chance I'll hire you. The last two hires whipped out apps they'd built and then we went over how they accomplished it. To me that's more valuable than any other process, having a portfolio or sample ready when you show up.
• freework 14y ago
  I've been interviewing pretty much non-stop since last April. I feel like I've become somewhat of an expert when it comes to interviewing processes. First off, as I've become a better programmer, I feel this fact is best shown in my opinions. When I first started out, if you had asked me my opinion of PHP vs Python, I would not have been able to say anything coherent. Now that I've worked with both technologies, my opinion is much more coherent. Instead of asking puzzle questions, ask candidates their opinion. "What is your opinion of Mongodb?" I've spent enough time in the trenches with Mongo to have quite an opinion of that technology. Same with Postgres, Django, Javascript, Coffeescript, etc. I think hands down, the worse type of interview questions are the ones where you're told to solve a problem, but you can only solve the problem in a certain way. Its become a interview idiom for me. One recently was to reverse the words in a string. In comes "This is a string", out comes "string a is This". No problem. In python this can be done with one line: the_string.split()[::-1] It takes me 5 seconds to write that out. The interviewer telle me "good job", then he asks me to do it again, but this time to do it without using any standard library functions such as split or reverse. At this point, I can almost with 100% certainty that this company will be a terrible place to work. It wouldn't piss me off it it wasn't for the fact that like 90% of all interviews I do end up like this. I've through about this a lot, and what I've come up with for an explanation as to why I can;t do these types of questions is because my algorithm writing process is very subconscious. When I'm writing code and I come to a problem that requires a lot of thinking, I usually stop, do something else and let the problem float around for a bit in my subconscious. I got my best ideas when I'm in the shower, on my bike riding around time, even while reading a news article. When people are watching me (especially strange people I don't even know) I don't do my best thinking. At this point I'd do anything to be able to think in front of people. Because its keeping me from being able to get jobs.
  • mattstreet 14y ago
    I think you're wrong about the 100% bad place to work just because someone asked you to reverse a string without using library functions. I would think your computer science education would have prepared you to plan out such an algorithm. Sadly, its pretty had for a place to confirm you have subconscious skills by asking a question and then waiting for you to come back the next day with the answer once it popped into your head.
• maigret 14y ago
  Kent Beck tweeted a very fitting comment: 'Programming contest problems shouldn't be algorithms, they should be like "set up continuous deployment of a multi-region Django app on AWS"' https://twitter.com/KentBeck/status/299528659746840576 https://twitter.com/KentBeck/status/299528659746840576
  • papsosouid 14y ago
    Why should programming contest problems be tests of practical sysadmin skills? That makes no sense at all.
    • maigret 14y ago
      Because this is much more than sysadmin skills involved. You've got to get a scalable application written first. It doesn't need to be complicated, but it needs to be robust and keep data consistent... etc. Also, this type of skills is badly missing to fresh grads.
      • papsosouid 14y ago
        No you don't, it says setup a django app. That involves writing no code, and is entirely a standard sysadmin task.
        • maigret 14y ago
          Script is code too. But I don't think you get there with plain sysadmin skills. How do you sync up the data between the regions? I have worked on a similar case not long ago, and there were definitely more than just "sysadmin" skills involved. I suppose Kent Beck also went through the same exercise. Pointing to the deficiency of usual contests is a good thing. In any case, if all sysadmins can do continuous deployment, cloud management and worldwide distributed apps, then I've definitely met the wrong ones before.
• pjmlp 14y ago
  What is this continuous reference to FizzBuzz from American developers? Being born in the other side of the Atlantic if it wasn't for HN, I would never heard of FizzBuzz in my 14 years of software development.
  • TheCoelacanth 14y ago
    I think it's just because of well-known blog posts[1][2]. I don't think it's a particularly common question for interviewers to ask here either. [1] http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html http://www.codinghorror.com/blog/2007/02/why-cant-programmer... [2] http://www.codinghorror.com/blog/2007/02/fizzbuzz-the-programmers-stairway-to-heaven.html http://www.codinghorror.com/blog/2007/02/fizzbuzz-the-progra...
• feliperibeiro 14y ago
  This is a complicated issue, but in my opinion the google/facebook style of interview is the one that gives you less false positives, even though it gives tons of false negatives. In other words, very few people who can do it are bad coders (false positives), but many good coders can't do it (false negatives). So, as companies like Google and Facebook have so many applications, they're much more concerned about false positives than false negatives.
• papsosouid 14y ago
  >I remember the first time I interviewed for a front-end programming position and got asked how to do something in JavaScript on a white board >how little it had to do with the actual job. That doesn't make any sense. How is solving a problem with javascript irrelevant to the job of a front end developer? That's what they do.
  • Ziomislaw 14y ago
    oh! you write your code on a whiteboard? do you have to type it into a computer by yourself or do you have people doing this for you? ;)
    • papsosouid 14y ago
      Writing it on a whiteboard instead of in a text editor takes it from 100% completely relevant to 100% completely irrelevant?
• j45 14y ago
  "Clever architecture increases possibility. Clever code increases complexity." - @edw519 the other day - https://twitter.com/edw519/status/301393887174483968 https://twitter.com/edw519/status/301393887174483968 Testing with excessively clever tests and puzzles will attract a mindset of creating more complex code and than needed. Architecture a problem in a way that solve the problem elegantly but remains open and flexible is far more a tougher challenge than an algorithmic challenge. There's a fine line between testing a hire to make sure someone's behind the steering wheel vs. communicating that you want things solved in interesting ways.
• libria 14y ago
  Key word here was "front-end". Algorithms aren't usually featured as often there. Before we break out the broad brushes, lets not paint all programming jobs as requiring the same skills. [1] Eric Lippert summed this up quite nicely: >> Does not having knowledge in Data Structures really affect one's career in programming? > Well it certainly will prevent you from getting a job on my team. But like I said before, programming is a huge field. There are lots of kinds of computer programming that don't require knowledge of data structures. [1] http://programmers.stackexchange.com/a/102123 http://programmers.stackexchange.com/a/102123
• pjungwir 14y ago
  I interviewed at several companies about a year ago, and one had a practice that really stood out. I was impressed by how organized it was, how well it seemed to be a good test of a job candidate, and how it gave me many opportunities to show my qualifications. It was a whole day, but they flew me out and put me up in a nice hotel nearby. The day was broken up into sessions of about 1 hour each (maybe a bit more). In each session it was me and 3-4 other people, but the people rotated, so I got to meet lots of engineers, a PM or two, the hiring manager, etc., and they all got to meet me. A few people showed up more than once. The first session was basically an introduction. I think it was shorter than the others. No quiz-style questions, but stuff about my background, etc., and opportunities for me to ask questions also. Next I did a presentation at the whiteboard describing some project I'd worked on. They had told me this would be a part of the day, so I could have prepared (though I didn't). I talked about my recent startup, describing some of the data model and machine learning tricks I'd developed. They asked lots of questions, some to clarify, others about my reasoning, how it would scale, if I'd considered technology X, etc. In the next session was me at the whiteboard again, but here they described a new feature they'd like to add to their site, and my job was to sketch out a rough solution. So partially this was about my ability to ask clarification questions, and also about my big-picture design skills. Again they asked lots of questions, and once we even got into a specific SQL query (where I opted to use an EXISTS with a correlated sub-query). So this session seemed a lot like the first, but more impromptu on my part and probably more practiced on theirs. Then they took the whole team out to lunch at a nice Thai restaurant (~15 devs plus the director of engineering and the PM). This part was relaxing and fun. Of course this was still part of the interview, and wouldn't be relaxing for everyone, but still it was a good way to feel out personality fit for both sides. Also there were too many people for me to be the focus of attention, so that was a nice break. Finally there was one afternoon session where they gave me a laptop with a toy Rails app, and had me add a feature (change a relationship from one-to-one to one-to-many, or something like that). There were two devs watching me code, and I had to do the whole MVC thing: write a migration, tweak the controller, edit the view. I got to talk through the changes as I did them, so they knew my reasoning. It made me smile to slip in a few fancy vim moves; I don't know if they noticed. But it made me realize this was also a test about some basic mechanics, too. They let me choose my editor/IDE, so it was a chance for them to watch whether I knew my tools. Then there was a final session for follow-up questions etc. This was fairly short I believe. All along the way I had lots of opportunities to ask questions of my own, which I appreciated. There were no puzzles, no trivia quizzes. Personally I'd add a FizzBuzz test to the initial phone screen (like Han Solo "They hardly asked me any questions."), but otherwise I plan to completely rip off this template the next time I hire someone myself. Maybe it can help some of you!
• at-fates-hands 14y ago
  This topic keeps coming around and it always makes me wonder why people have such a hard time hiring good programmers. I say because the best places I've worked at already have a "system" in place for their front-end people. You can still be a marginal developer and do well if you follow the system. It takes a few months, but once you know how they write their code and the standards they use, it's pretty easy to write sustainable, reusable code. This also makes it considerable easier to think and work outside the box. You can take a few of the better programmers, tell them about a new framework or approach you want to use on a project and let them loose. If it's a success, you merge these new ideas into the existing ecosystem. Simply repeat as necessary. The industry really puts a lot of hard work into finding and hiring good candidates, but it's not the programmers who are important. It's the system you're plugging them into which matters.
• soseng 14y ago
  To change it up a little bit, has anyone here been asked simple debugging questions? I've interviewed at numerous places and I've never been asked the question why a certain piece of code is breaking or how to fix it. The questions could be designed to be self-contained and just complex enough to get a feel for how the candidate thinks. My consulting job usually takes me to places where a project is in trouble or failing. It would seem that quickly understanding broken foreign code would be a huge asset. The closest question I've been asked is: "Here's a Singleton implementation in Java. What can go wrong?"
  • suresk 14y ago
    I tend to think debugging is a pretty useful indicator of how well someone will do - the best developers I've known were really good at debugging, and I've seen very few people be good at debugging and bad at programming - so I usually try to incorporate a debugging question or two into my interviews. I like to ask debugging type questions in sort of a role-play format. Usually I'll start with a symptom that an end-user might see and then allow the candidate to ask me for any additional information (as either the end user or a system admin giving them log files and other things) to help them narrow down what the problem might be. It is a little tricky to make it possible for them to make progress without knowing anything about the hypothetical code base, but I like it because I think troubleshooting problems is a decent indicator of someone's skill and knowledge: Do they have a good approach to narrowing down where the problem is? Do they know what kind of information to ask for (for Java devs, I like to ask questions that lead them towards asking for thread dumps or heap dumps), which shows mastery of a language/platform? Do they ask good questions in general? I've also seen people use printed stack traces and ask the candidate walk through what the stack trace is telling them and what the possible root causes might be - this is more useful when you want to test someone's knowledge of a specific framework. Handing someone an application in a broken state and watching them work through the process of debugging it would seem like it would be interesting, but I've never had the time to prepare this for an interview.
• zenbowman 14y ago
  It's one thing to ask people to right syntactically correct code. It's another thing to ask them to write pseudocode in whatever language they please and understanding their process. At the end of the day, as Hal Abelson said, computer science is not about computers, or science. It's about being able to formalize process.
• megaframe 14y ago
  I wish this were the case more. I graduated with and electrical engineering degree, but over the years I've spent the bulk of my time coding up DB, big data stats, ML, etc. to help me do my job. Trying to explain I'm more than capable while I falter with simple puzzle based question is an impossible gauntlet that has kept me from even considering a career shift. (Granted I'm a much better analog designer than I am a java code.)
• garraeth 14y ago
  Comments are closed on the article and I wanted to thank David. I'm one of those who freeze up like a deer in headlights when asked to do a quiz. But have a degree in CS, have made several enterprise level packages by myself, and have been programming for 31 years -- so am fairly confident that I can program.
• saman_b 14y ago
  That's what I was discussing with my friend a few days ago. He had an interview with Google last month and it took him a month or two to prepare for that. he did not pass the interview, but he got an interview with Amazon this month. He was busy with other things and he could not study for the interview. He mentioned that he cannot remember most of the things he prepared and read for the last interview. I am forgetful too, and I guess 90% of people forget details about algorithms and how to answer a puzzle very fast if they don't use it everyday. So the question is what is the value of a skill or knowledge that fades away after a month or two? I know people who got hired by Google or other big companies that forgot all the information they acquired before the interview. Cause most of it is useless for the actual job. Many logic questions or puzzles are well known and people tend to know the answer before the interview. If I had to interview someone; I would assign him a project from the company that is part the of the job he is going to do, leave him alone in a room with a computer for a time period I expect someone to finish that specific project. Finally, I come back to see the out come. It is good for the applicant, as there is no pressure on him (except some time constraints maybe which is not a big deal), I do not put him on the spot and he does not have to deal with his boss while answering to a question. He does not have to dig unrelated information before the interview, and he can show off his programming skills if he has any. It is good for the company, as I do not have to spend a day interviewing and prepare for that, I will know right away from the code if the guy is a good fit or not, I can see his programming skills, I know ahead of time how long in average it takes for an internal person to do the job, so I can measure his performance relatively. I do not even have to evaluate the code in front of him, I can invite multiple candidates at the same time, put them in separate rooms. Get the results, evaluate them and invite them for the second round if I liked their code.
• angular 14y ago
  Trying hard to be pithy and failing miserably, must be a 37singnals post.
• laureny 14y ago
  Congratulations to 37signals for adopting a hiring practice that most companies have been using for more than a decade.
• codex_irl 14y ago
  I like this: http://cdnnew.theinspiration.com/wp-content/uploads/tattoo_QR.jpg http://cdnnew.theinspiration.com/wp-content/uploads/tattoo_Q...
• mcphilip 14y ago
  The last interview I gave for a senior dev I tried making up some custom discussion questions to give after the basic tell me about yourself stuff. The questions boiled down to: 1. Junior developer code review where a save method included a call to a list method "because it's more efficient" than making a separate ajax call. Explain to the junior developer why that's not necessarily a good idea. 2. UI wants a tree view of categories where each category has a name and a parent category. The number of categories can be arbitrarily large. Discuss problems you might run into using a database agnositic approach (e.g. Hibernate) and what solutions you'd propose. 3. Prototyping a new app, two potential clients in the same domain have widely different ideas of how those domains should be represented (i.e. obviously similar domain class names, very different properties). Discuss ways of modeling domain classes such that each client sees the data in their preferred format. All in all, I think the interview went pretty well because the discussion questions led to discussing all sorts of related topics that gave me a good feel for the types of problems the candidate had run into and if he had success in dealing with them. It was great giving an interview without arcane things like explain the yield keyword in C# and how it might be used.
• monksy 14y ago
  On Coding Portfolios: http://programmers.stackexchange.com/questions/102490/requiring-programming-portfolios-in-interviews-recruiting http://programmers.stackexchange.com/questions/102490/requir...
• webo 14y ago
  I was interviewing with Microsoft last semester for a SE internship. At the end of the interview with a senior engineer, we were just chatting about various topics. This is one topic that we talked about. His response was something along these lines: "Of course we do not write code on whiteboard everyday. Of course we do not write code to detect if a graph is acyclic (my interview question) on a daily basis. Of course we do not put our engineers on the spot with these types of questions. However, when the time comes, may it be only once in 10 years, we know that we have hired the right people because these questions show your thinking abilities - not if you know the answer or not." Btw, I didn't get an offer, but I'm going to Amazon where I had very similar interview process as Microsoft.
• spiralpolitik 14y ago
  Programming is about problem solving. What you want in a programmer is someone who knows how to approach solving a problem in a structured repeatable manner. What getting someone to solve small problems like FizzBuzz in the interview gives you an idea of how then approach the problem. Can they decompose the problem into smaller chunks ? Can they then refine their solution into a better solution ? Do they know when to stop refining before the solution becomes difficult to understand ? These are the skills you want to asses from a potential programmer. Programming languages, platforms, frameworks come and go but if you don't know how to approach solving a problem then that knowledge is ultimately worthless.
  • dpcan 14y ago
    I totally agree, but only in the FizzBuzz case. It's the perfect, super-simple puzzle that can be solved quickly, but let's you see how a programmer might approach a problem, how they like to code, and their go-to language, etc. It's not like you are trying to stump the programmer. Any programmer with a lick of skill should be able to knock out the FizzBuzz problem in their language of choice. It's actually kind-of fun the first time you do it. Personally, I'd watch the programmer's face. If they smiled after realizing the problem is not as clear-cut as it seems, they probably enjoy programming.
• bjoe_lewis 14y ago
  Ok, this is true story and has nothing to do with my credibility. Few days ago glenwood systems, a core product developing company conducted a recruitement drive in my college. There was me, a coding geek with all my passion for one thing, and there was him, who is a puzzle freak. They selected him past the first two rounds full of puzzles and math, finally rejected him knowing his programming incompetence. And they missed someone who really could help. me. There are a ton of books out there teaching you how to crack interview puzzles and programming riddles. There's no book telling you to get your hands dirty with projects and code. The author is right. Hire the one who did things, not the one who learnt to crack puzzles.
• michaelochurch 14y ago
  I'm going to avoid the cultural flamewar around whether companies should do this. I don't mind it. I've interviewed for enough jobs and had enough acceptances and rejections to learn something: it's not a big deal. If you don't know an API function, you don't know it. The likelihood that you won't get the job because you didn't know that one question is unlikely. In fact, part of the evaluation is whether you can handle not knowing the answer with grace. The fail-outs are the ones who use their cool or seem to think the question is stupid, not the ones who get it wrong. Once you realize that you don't need to get all of the answers to pass an interview, it gets easier.
• geebee 14y ago
  One of my career goals is to get a programming job without having to show that I can add a leaf to a binary tree through recursion (again). I don't mean this as flippantly as it might sound. I used to think it was kind of fun and exciting to be at the whiteboard, thinking on my feat, dealing with various curveball data structures and algorithms questions. I certainly always read up on this stuff prior to interviews, and from what I've heard, I'd be advised to do it again if I get an interview at google or something. But the last time I did this left me depressed. I realized that I've done so much of my work in obscurity that people are still asking me about binary trees. I want to be very clear that I don't blame them! I also started to realize that I have a lot more to offer than my ability to traverse various tree structures, and that the interviewers seem unaware of this. But it's up to me to show them, not on them to just take me at my word. The last time I went through a massive, all day technical interview, I didn't get the job, partly because I didn't review my data structures and algorithms book (again), but probably also because I came off as cranky and irritated. I was busy that week, probably should have put it off. Just earlier that week, I had taken several clients from pharma and semiconductor companies through some beta testing of our software, refining the model, getting feedback on use, optimizing the code base so it could give answers more quickly. The company I was interviewing for created supply planning, forecasting, and inventory management software for various large businesses. As a math major with an MS in Industrial Engineering and a background writing software, it was right up my alley. My interviewers showed no interest in anything other than my ability to code or a couple branches of math. In fact, some of them seemed so inexperienced that I'm not sure they were aware that this kind of experience even exists to be asked about (Oh, you work with clients? Our project managers do that). At lunch, one guy asked me how to swap two integers without creating a third integer. This is after a full morning of technical grilling on math and programming, with a lengthy afternoon to follow. Fortunately, I'm now in a job where I am encouraged, not just allowed, to contribute to open source, do presentations at conferences, try to build communities, and so forth. I would not consider any job that did not have this element (and now that I have a job like this, I'm not looking anyway). But if I were to look around, I would vastly prefer to be hired for my known and widely used code base than for my ability to detect cycles in linked lists, implement a hashing function, or print out all possible permutations of a string (with no duplicates!). I'm not there yet, but that's my career goal.
• russelluresti 14y ago
  Just my 2 cents on this issue - writing code on a whiteboard isn't always about the code you write. It's not about the actual code you produce in the interview, it's about talking through your thought process, your ability to reason while under pressure, and your attitude of whether or not "this is beneath me - wah!". I've been in several interviews where I had to write code. I made some mistakes, some things I plain out didn't know (for my first job, I had no clue how to write any JavaScript, and I explained that it was just something I didn't know how to do, but that I'd be willing to learn it as required). I still got hired from all of them. So, if you think coding tests during interviews are about producing flawless code, they're not. They're to see how you think - and that is profoundly important as a developer. I would also argue that a developer's existing skills are the least important factor when deciding whether or not to hire someone. Every developer works as part of a team and has to collaborate and communicate with others in that team. If you want to hire a developer, you need to see how well they communicate complex technical problems to non-technical people.
• asmosoinio 14y ago
  Really liked the text at the bottom: (If you need help posting a comment, feel free to use any of these samples: “You make todo lists, you don’t need real software engineers”, “Math is actually really important, you know!”, “Google is worth one gajillion dollars and they use quizzes, so there!”)
• mattbillenstein 14y ago
  Most eye-opening thing about programmer interviewing I've read: https://sites.google.com/site/steveyegge2/five-essential-phone-screen-questions https://sites.google.com/site/steveyegge2/five-essential-pho... I probe for weakness in 5-6 different areas of computer science in the phone screen - keeps me from bringing in weak candidates. Then in the on-site, I give them my Macbook and ask them to solve a few simple problems with some flat files I conjured in a language of their choice that's installed on that machine (C/C++/Java/Python/Ruby/PHP/Perl/shell/etc). You get an idea of how they solve problems, think about solutions, fix bugs, avoid common errors, and what they look up on the web (I let them use a browser to lookup apis and so forth - it's a good way to weed out "cut and paste" programmers). The people who do well finish quickly and we chat about the company with the remaining time - the people who do poorly use almost all the time on the first problem, make a bunch of mistakes, and usually don't understand the proper data structure for the job.
• sippndipp 14y ago
  Agreed - but I guess if you're 37signals you just attract the right people and you don't have to worry at all. But if you're not then maybe some of these riddles are just good heuristics.
• jdavis703 14y ago
  At my old company we'd tell interviewees to bring a laptop loaded with there usual dev tools. After a couple of typical interview questions they were then instructed to create a small program using whatever language/tools they'd like. They had "unlimited" time and sat at an empty cubicle. Candidates also has had full access to the Internet. It was amazing how several people could not even complete the simplest program without needing extensive assistance.
• Tekker 14y ago
  When I interview others, I use a couple of basic whiteboard show-me pseudocode questions (how would you reverse a string, how does a linked list work, give me an example of a callback). It seems to me that if they know their stuff, they can answer those. If someone can't get through any of them, or at least have a valient stab at it, chances are they don't have the background I want. By the same token, I've been interviewed myself where I've been given weird shit questions like others have mentioned; not the blender one, but shit along that line. That's nothing I personally do without some serious contemplation, not to be achieved in 5 minutes in an interview. Anyone who does so likely knew the answer ahead of time or is extremely puzzle-oriented. I think the real test is to examine the puzzle-solving processes of the candidate, but as a rule I don't think much of those kinds of questions.
• edwardunknown 14y ago
  I'd never work for a place that gave me a fucking puzzle to solve. I'd probably smack the interviewer upside the head for wasting my time. The best way to do it is pay the person per task as an independent contractor until you're sure a full time position is what you're both looking for. If they're helpful to you then you keep calling them and if not everybody keeps their dignity, no big deal. Don't humiliate potential employees right off the bat with stupid games.
• lucian303 14y ago
  Amen. Couldn't agree more. Lots of companies miss out on great engineers. Companies that do hire on this tend to have bad to mediocre engineering and leadership and no ability to actually tell a good engineer from a bad one.
• sytelus 14y ago
  Some of the best hiring strategy I've came across is indeed "test driving". Take a challenging problem your team has, slim it down and ask candidate to do some quick prototype during a weekend. No interviews, puzzles or whiteboarding :). If you were going to fly down a candidate on-site for a day of interviews instead, it's going to take same time anyway considering logistical effort on candidate's part. Test-driving, however, is unfortunately neither scalable nor "leak proof". As soon as one candidate gives away your test drive question, the next candidate could easily cheat away inflating their apparent awesomeness compared to other better candidates. However it would be incorrect to write down all whiteboarding interviews as "evil". Like any other interviewing techniques, it really depends on how you do it. A good whiteboarding question that allows candidate to use CS fundamentals to solve fairly non-trivial problem that you also needed to solve for your current work is a great question. There are 3 major reason why this doesn't work as expected at some large companies: 1. Interviewers can't think of good whiteboarding question and fall back to commonly asked popular puzzles. These interviewers are also often the ones who have one "favorite" question that they would ask everyone. This is absolutely #1 problem why whiteboarding interviews devolves in to secret puzzle marathon. The best questions that interviewer could ask are actually the ones that they needed to solve themselves as part of their work recently. This keeps questions relevant and refreshed regularly. It also allows to compare candidate with themselves and follow the philosophy of hiring people smarter than you. It's however hard because interviewers themselves needs to continue doing something interesting regularly. 2. Interviewers provide feedback to each other during the loop. It's not coincidence that if 1st interviewer gives "hire", all rest tend to do same at companies where there is no clear policy of not communicating feedback before end of the interview. 3. Interviewers don't follow up candidate response by extending question, building on to next level perhaps open ended scenario, asking auxiliary details such as test cases, complexity etc.
• ricardobeat 14y ago
  Aaand HN has just written a 10-hour book on the subject of hiring.
• DigitalSea 14y ago
  I agree wholeheartedly here. I know many developers (myself included) who would fail a FizzBuzz test or any instance where they're asked to write code on a whiteboard and yet can produce some really clean, efficient and above all well-written code. How about instead of asking people to write on a whiteboard or do Computer Science 101 tests, how about you give them a computer, a problem and ask them to solve it? Don't forget it's not always how smart they are, it's their work ethic and whether or not they'll be a good fit for your team. Personalities that meld well with already hired employees are an essential must, no point hiring an introvert for a position when the team are extremely social people who have Friday afternoon beers. Here is a tip: hiring a developer? Ask them to write a blog application, a task management system or a real world problem they will encounter if hired by your company not some ridiculous test that could stop you hiring a great developer. I consider myself to be pretty good, but I don't have a CS degree, I suck at mathematics and can only do simple math, yet I am able to produce exceptionally great code especially in instances where deadlines are tight and sometimes unreasonable.
  • suresk 14y ago
    I'm sympathetic to this - I don't have a CS degree and most of my last jobs have been through references from people I've worked with before, so I haven't had to answer any tough programming questions in an interview in quite a while. I'm sure if I tried to interview at Google, Amazon, Microsoft, etc, it would be a little tough for me. At the same time - any reasonably experienced developer should be able to whiteboard something as simple as fizzbuzz without a problem. The more complex the problems, the more you worry about the whiteboard effect, but you should be able to at least walk through your thought process and come up with some reasonable attempt. Having a candidate build something outside of the interview process probably isn't bad (as long as you follow up by going through the code with them), but I think something like a blog application is both too big (unless you really constrain the feature set) and too simple to work in some cases. Something that requires a bit more thought and a bit less time would be better, IMHO.
• agentultra 14y ago
  I have a long-winded blog post about this very subject awaiting some polish. I've done my fair share of interviews. I still don't like them. I've narrowed it down to basically two things: 1. There is a lot of highly subjective, mystical advice about what "works," in a technical interview and what doesn't. Big problems, small problems, trivia, puzzles, white-boards, no white-boards... hardly any of it is thoroughly studied and built on solid theory. It's all conjecture and personal bias. 2. How the candidate is feeling and where their head is at on the day of the interview matters. I can go on about cache alignments, bus errors, segmentation faults, C++'s lack of order in evaluating function arguments, why template class declarations must exist in the same compilation unit as the definition, why CLOS is amazing and the computation model of recursion is brilliant (or at least why I think so). Yet on a bad day I can (and have) botched perfectly trivial, benign problems. However at the end of the day you need some sort of process. I just think that the current practices are not good enough. It seems to me that companies are probably turning away perfectly fine candidates without realizing it. There's pros and cons to every approach I suppose. But I still think trivial problems are dumb.
  • suresk 14y ago
    I agree that the hiring situation in our industry is less than ideal. The irony is that, for all of the data we use and tools we make for others, our hiring practices often rely on few tools, limited data, and lots of gut feelings about what is important. Take the process of advertising a position and moving candidates through the hiring process - the tools there are almost universally crappy and haven't made a lot of progress in the last decade. Also, as far as I can tell, very few companies attempt to record any kind of data about how candidates are hired and how they subsequently performed on the job (even tracking job performance is a tough problem) - and it certainly doesn't exist industry-wide. There is a ton of money in this area - why hasn't there been much progress? Is it because it is an unsexy problem, or is it because it is genuinely impossible?
• neeraj_r 14y ago
  Thats good. So we don't need anonymous to crack your system.. :)
• deleted 14y ago
  [deleted]
• known 14y ago
  interview != quiz
• alan_cx 14y ago
  Not employed people for years, but I never put much value on ultimate skills. I used to think long term. So, give me the the right human being, with a reasonable level of knowledge, and very quickly they will be more than good enough. A little more time and for me, they become pretty much perfect. I want the right personality. The ultimate skills will follow. So, I just chat to candidates and employ those who I feel I and my people can work with. I never left it so late that we needed instant skills. To me that would be poor management. Prior to that, I have tested and done many of the things people cite here, but most of the time I got awful employees who turned out to be good at doing interviews. Yeah, a bit hippy dippy, but it worked for me.