4 ms·

I've never understood the use of puzzles and tests for hiring. I'm notoriously bad at them, but I consider myself to be a very solid programmer, with 13 years

I've never understood the use of puzzles and tests for hiring. I'm notoriously bad at them, but I consider myself to be a very solid programmer, with 13 years professional web development experience, and a major open source project under my belt. I think one of the reasons I don't do well on these is because my brain doesn't seem to work well for hypotheticals. I need real world use cases or else it's exceedingly difficult for me to work out a solution for a problem that is too abstract. Maybe I'm unique amongst other programmers with my level of experience. It's possible that other programmers love doing puzzles like this, but I tend to get discouraged if I'm looking at a company, and they use tests and puzzles to gauge my capabilities. Especially (and I wouldn't accuse Facebook of this, but it happens much too often), if those tests and puzzles are substitutes for looking at the massive amounts of code I've written.

endtime 16y ago
I think the attitude of a lot of companies, especially the ones hiring fresh grads, is that the ability to think is more important than the ability to code (the latter of which can be taught far more easily than the former). I don't mean to imply that you are unable to think, of course - the puzzles are surely an imperfect heuristic - but that's my understanding of their aim.
deadc0de 16y ago
The Facebook puzzles are pretty easy. That being said, I hate puzzles too. I think it's pretty degrading to ask a well-established researcher/engineer to do that kind of stuff. On interviews I typically ask people to give a talk about their previous projects in detail. A lot of nuances arise during these talks that allow you to quite easily judge the level of the candidate.
- hammerdr 16y ago
  Why is it degrading? I would think it would be perfectly acceptable for companies to ensure that their new hires (experienced or not) pass their basic heuristics. This comes in the form of tests, resume checks, social compatibility, etc. (I ask this from the perspective of a young, inexperienced developer.. so excuse the thoughtlessness!)
  - deadc0de 16y ago
    I don't know... I would personally feel a bit weird asking, for example, an experienced compiler developer with a CS PhD to reverse a linked list, or words in a string, or something like that.
    - soamv 16y ago
      So get them to solve something more difficult than reversing a linked list. A good programmer will enjoy a good puzzle, especially if it is new to them.
prostoalex 16y ago
Most of the time anyone with solid engineering background, industry experience, programming contest medals, or entrepreneurial streak would have no problem landing an interview. However, that leaves a whole lot of candidates out of the picture. Faced with limited recruiting resources, puzzles are about the next best thing to serve as a recruiting filter. Plus some people send them in without being told to, which is usually a plus, as it implicates coding in their spare time is something they enjoy doing.
bbarthel 16y ago
Please don't ask me to review your "massive amounts of code" - I guarantee you will not come out ahead in that evaluation, if I even have time to properly evaluate it (according to Code Complete, you can properly review code in a high-level language at about 100-500 lines per hour). If you consider the fact that: - It is unlikely I am well-versed in the problem domain of your code, - I am unlikely to be aware of external design constraints or coding pressures, - I am probably going to randomly select the worst section of code to review, - I have no context for how the code evolved the chances that I walk away with a negative impression are high enough that it should give you pause. If I do manage to walk away with a positive impression of your code, it will be offset by the perception that you believe your time is more valuable than mine (why else would you send me a massive amount of code and ask me to review it to determine your suitability for a position) - and you still haven't answered the other big question I need answered which is whether you are a good "fit" for the team from a personality/cultural perspective. If you really want to impress me, go ahead and send me a list of your projects and contributions and then slice a relevant sample of code from it. Describe what the problem was and how you solved it so I can understand the context. Try to make it relevant to the position, but failing that describe how you could utilize that experience in the relevant domain. Keep it small enough that I can review it in under an hour or two. Is it a lot more work for you? Absolutely, but if you do this I guarantee you I will look at your code and will likely call you for an interview because you are demonstrating a high level of competence and professionalism while still respecting my time and needs.
- michaelchisari 16y ago
  You're assuming that I would "send you" massive amounts of code. I wouldn't. I'd have a link to my github on my resume, along with a description of the projects I've worked on. It's presumptuous that I would be asking you to do a full code review (my Appleseed project is tens of thousands of lines of code, for instance), and that I wouldn't recognize the impracticality of such a request. I would find it problematic, then, if you responded with tests and puzzles, without any interest or questions about my code. Or if the test and puzzles couldn't be skipped once I mentioned my experience level and accomplishments. This is often what interviewers have done, and I find it to be a good indicator of a workplace I'm not interested in working in. The best workplaces let their developers take some time to comb through the best parts and the worst parts, and have a conversation about what I've done, my goals, how I did things, etc.
bfung 16y ago
How are these puzzles hypothetical? They look better written in specification than specs in the real world: (selected few and summarized...) Hoppity Hop! Input: filename as arg from command prompt, file contains a number Output: Some variation on the word "Hoppity" repeated Simon Says the gist: Connect to some server, call some api's in a certain order. Battleship - write a client that connects to some server that plays battleship... Find Sophie - read some data,build some data structures/run algorithm,output result Input: filename and specifications of the file format Output: result Some test simple programming skills (Hoppity Hop is basically the Fogbugz for-loop question + no brainer file parsing) and the harder ones seem to involve knowledge of (not too tough) algorithms. If the questions were more like "How would you move Mt. Fuji", then that's certainly hypothetical.
- michaelchisari 16y ago
  Hoppity Hop is basically the Fogbugz for-loop question + no brainer file parsing Why not give me a real world use case, with a code base to work off of? Put up a codebase on github, provide a set of user stories and requirements to satisfy (ie, "add a real time search using this data set and code base which works cross-browser and degrades in IE"), and ask me to add it, thus basically mirroring a normal development process. As someone who has had to interview programmers before, that would tell me infinitely more than their ability to re-iterate the word "Hoppity" on the command line.
  - wh-uws 16y ago
    "Why not give me a real world use case, with a code base to work off of?" Because they want you to be good enough to do it without that. (I'm saying this as an about to be new grad who is studying these subjects to not have that much help and have to do it on a whiteboard in an interview and also working on these puzzles and similar ones from a lot of other companies)
    - michaelchisari 16y ago
      As someone with a lot of industry experience: It's great to know how people solve puzzles and tackle algorithms. It's even better to know how they can fit those solutions into a bigger picture, how they structure, comment and document their code, and how they interpret a use case and user stories. Dealing with things in the abstract, stripping it of context, interface, interaction, and real world requirement, seems to be an incomplete way to understand the capabilities of the person you're potentially hiring. Unless, of course, you're hiring for the position of Junior Abstract Puzzle Solver. In which case, by all means, Godspeed.
egor83 16y ago
These FB puzzles test your algorithmical capabilities; and being able to develop new algorithms is not always necessary (though always useful) for a programmer. So you can have lots of programming experience and still not be too good at algorithms. But it is important for Facebook - so that's what they test.
dancavallaro11 16y ago
The reality, for better or for worse, is that a lot of the companies that are using puzzles like this prefer false negatives over false positives. In other words, they would rather pass over the occasional excellent engineer than accidentally hire a mediocre engineer. This is the case at Google, Amazon et al, and likely at Facebook and others as well. It very well might be the case that you're just an unfortunate casualty of this sort of hiring practice.
- michaelchisari 16y ago
  I think you hit the nail on the head, but here's my issue: It's not so much companies like those that have a problem. I've been recruited by all the big guys on the strength of my open source work (save for Facebook, but that's probably due to a major conflict of interest, considering what I'm working on), but the real problem arises when you have smaller companies latching on to this approach, without the recruiting power of a Google or an Amazon. So now I get contacted by a small startup who all of a sudden wants me to take a quiz because they fancy themselves the next Google.
jamii 16y ago
Well, its certainly more fair than giving puzzle/coding questions during an interview. This way you are under no pressure and can work on your own machine in your own time.

endtime 16y ago

I think the attitude of a lot of companies, especially the ones hiring fresh grads, is that the ability to think is more important than the ability to code (the latter of which can be taught far more easily than the former). I don't mean to imply that you are unable to think, of course - the puzzles are surely an imperfect heuristic - but that's my understanding of their aim.

deadc0de 16y ago

The Facebook puzzles are pretty easy. That being said, I hate puzzles too. I think it's pretty degrading to ask a well-established researcher/engineer to do that kind of stuff. On interviews I typically ask people to give a talk about their previous projects in detail. A lot of nuances arise during these talks that allow you to quite easily judge the level of the candidate.

hammerdr 16y ago
Why is it degrading? I would think it would be perfectly acceptable for companies to ensure that their new hires (experienced or not) pass their basic heuristics. This comes in the form of tests, resume checks, social compatibility, etc. (I ask this from the perspective of a young, inexperienced developer.. so excuse the thoughtlessness!)
- deadc0de 16y ago
  I don't know... I would personally feel a bit weird asking, for example, an experienced compiler developer with a CS PhD to reverse a linked list, or words in a string, or something like that.
  - soamv 16y ago
    So get them to solve something more difficult than reversing a linked list. A good programmer will enjoy a good puzzle, especially if it is new to them.

prostoalex 16y ago

Most of the time anyone with solid engineering background, industry experience, programming contest medals, or entrepreneurial streak would have no problem landing an interview. However, that leaves a whole lot of candidates out of the picture. Faced with limited recruiting resources, puzzles are about the next best thing to serve as a recruiting filter. Plus some people send them in without being told to, which is usually a plus, as it implicates coding in their spare time is something they enjoy doing.

bbarthel 16y ago

Please don't ask me to review your "massive amounts of code" - I guarantee you will not come out ahead in that evaluation, if I even have time to properly evaluate it (according to Code Complete, you can properly review code in a high-level language at about 100-500 lines per hour). If you consider the fact that: - It is unlikely I am well-versed in the problem domain of your code, - I am unlikely to be aware of external design constraints or coding pressures, - I am probably going to randomly select the worst section of code to review, - I have no context for how the code evolved the chances that I walk away with a negative impression are high enough that it should give you pause. If I do manage to walk away with a positive impression of your code, it will be offset by the perception that you believe your time is more valuable than mine (why else would you send me a massive amount of code and ask me to review it to determine your suitability for a position) - and you still haven't answered the other big question I need answered which is whether you are a good "fit" for the team from a personality/cultural perspective. If you really want to impress me, go ahead and send me a list of your projects and contributions and then slice a relevant sample of code from it. Describe what the problem was and how you solved it so I can understand the context. Try to make it relevant to the position, but failing that describe how you could utilize that experience in the relevant domain. Keep it small enough that I can review it in under an hour or two. Is it a lot more work for you? Absolutely, but if you do this I guarantee you I will look at your code and will likely call you for an interview because you are demonstrating a high level of competence and professionalism while still respecting my time and needs.

michaelchisari 16y ago
You're assuming that I would "send you" massive amounts of code. I wouldn't. I'd have a link to my github on my resume, along with a description of the projects I've worked on. It's presumptuous that I would be asking you to do a full code review (my Appleseed project is tens of thousands of lines of code, for instance), and that I wouldn't recognize the impracticality of such a request. I would find it problematic, then, if you responded with tests and puzzles, without any interest or questions about my code. Or if the test and puzzles couldn't be skipped once I mentioned my experience level and accomplishments. This is often what interviewers have done, and I find it to be a good indicator of a workplace I'm not interested in working in. The best workplaces let their developers take some time to comb through the best parts and the worst parts, and have a conversation about what I've done, my goals, how I did things, etc.

bfung 16y ago

How are these puzzles hypothetical? They look better written in specification than specs in the real world: (selected few and summarized...) Hoppity Hop! Input: filename as arg from command prompt, file contains a number Output: Some variation on the word "Hoppity" repeated Simon Says the gist: Connect to some server, call some api's in a certain order. Battleship - write a client that connects to some server that plays battleship... Find Sophie - read some data,build some data structures/run algorithm,output result Input: filename and specifications of the file format Output: result Some test simple programming skills (Hoppity Hop is basically the Fogbugz for-loop question + no brainer file parsing) and the harder ones seem to involve knowledge of (not too tough) algorithms. If the questions were more like "How would you move Mt. Fuji", then that's certainly hypothetical.

michaelchisari 16y ago
Hoppity Hop is basically the Fogbugz for-loop question + no brainer file parsing Why not give me a real world use case, with a code base to work off of? Put up a codebase on github, provide a set of user stories and requirements to satisfy (ie, "add a real time search using this data set and code base which works cross-browser and degrades in IE"), and ask me to add it, thus basically mirroring a normal development process. As someone who has had to interview programmers before, that would tell me infinitely more than their ability to re-iterate the word "Hoppity" on the command line.
- wh-uws 16y ago
  "Why not give me a real world use case, with a code base to work off of?" Because they want you to be good enough to do it without that. (I'm saying this as an about to be new grad who is studying these subjects to not have that much help and have to do it on a whiteboard in an interview and also working on these puzzles and similar ones from a lot of other companies)
  - michaelchisari 16y ago
    As someone with a lot of industry experience: It's great to know how people solve puzzles and tackle algorithms. It's even better to know how they can fit those solutions into a bigger picture, how they structure, comment and document their code, and how they interpret a use case and user stories. Dealing with things in the abstract, stripping it of context, interface, interaction, and real world requirement, seems to be an incomplete way to understand the capabilities of the person you're potentially hiring. Unless, of course, you're hiring for the position of Junior Abstract Puzzle Solver. In which case, by all means, Godspeed.

egor83 16y ago

These FB puzzles test your algorithmical capabilities; and being able to develop new algorithms is not always necessary (though always useful) for a programmer. So you can have lots of programming experience and still not be too good at algorithms. But it is important for Facebook - so that's what they test.

dancavallaro11 16y ago

The reality, for better or for worse, is that a lot of the companies that are using puzzles like this prefer false negatives over false positives. In other words, they would rather pass over the occasional excellent engineer than accidentally hire a mediocre engineer. This is the case at Google, Amazon et al, and likely at Facebook and others as well. It very well might be the case that you're just an unfortunate casualty of this sort of hiring practice.

michaelchisari 16y ago
I think you hit the nail on the head, but here's my issue: It's not so much companies like those that have a problem. I've been recruited by all the big guys on the strength of my open source work (save for Facebook, but that's probably due to a major conflict of interest, considering what I'm working on), but the real problem arises when you have smaller companies latching on to this approach, without the recruiting power of a Google or an Amazon. So now I get contacted by a small startup who all of a sudden wants me to take a quiz because they fancy themselves the next Google.

jamii 16y ago

Well, its certainly more fair than giving puzzle/coding questions during an interview. This way you are under no pressure and can work on your own machine in your own time.