15 ms·
One hiring filter that works
- LaaT 15y agoI wonder how much of this is due to novelty factor. I doubt that one could reach such good numbers, if asking for solving puzzles becomes a common practice.
- alan_cx 15y agoAnd perhaps tedious.
- K2h 15y agoThere is something to be said for the novelty probably getting more interest, but I think that giving puzzles around the real work the hire will perform tells them as much about your company and the problems they can help you solve, as it tells you about the candidates ability to do so. A win/win as they say. I already ask about 10 technical questions in each of the interviews I conduct, and having a problem online for each candidate to hack on would be really useful. Now that I think about it, I may even consider using something like this to drive internal improvement projects. Some people just love the challenge, and the 'a' people always step up, you just have to ask.
- deleted 15y ago[deleted]
- ceol 15y agoDo you have a link to your encrypted application?
- yummyfajitas 15y agoI don't know if it's still up, and I don't work there anymore.
- DugFin 15y agoYes, this is a bit annoying. First he relates a not entirely interesting story about his HR problem, then says he solved it by including an interesting puzzle. OK, cool story bro, NOW WHERE'S MY PUZZLE?!?!? I'm not looking for a job, or looking to hire, but like most Hacker News readers I like puzzles. Now I'm angry and have to find a puzzle somewhere else.
- yummyfajitas 15y agoMy previous blog post (submitted here yesterday) has two. http://www.chrisstucchio.com/blog/2012/leaving_academia.html http://www.chrisstucchio.com/blog/2012/leaving_academia.html But I'd suggest that except in special cases, posts with titles like mine are probably not worth reading if you are uninterested in hiring or finding a job.
- DugFin 15y agoWell, you never can tell. When someone says they've found a novel test for weeding out bad applicants, that sounds interesting. When the test is described with a handwave as an interesting puzzle that the best applicants leap at the chance of solving before reading the rest of the interview.... and then you don't even give us the puzzle, but rather launch back into the HR aspect... well, that just seems cruel. It's like reading a murder mystery where the detective details how he figured out who the murderer is, and then never tells anyone. From my authoritative position as a random anonymous internet jerk, I assert that if you allude to a puzzle, you ought to at least include it. I am apparently puzzle obsessed today. Slow day at work.
- ceol 15y agoI just bought that C++ book. Thanks!
- mchouza 15y agoThe puzzle can be found at http://meshcapital.com/application.tar.gz http://meshcapital.com/application.tar.gz
- 15y ago
- simonbrown 15y agohttps://news.ycombinator.com/item?id=1659735 https://news.ycombinator.com/item?id=1659735
- AndrewDucker 15y agoProviding you're testing the actual skills that you want, this works well. But if you're throwing out maths puzzles when you actually want people to write Java business logic then I can see you ending up with the wrong people.
- yummyfajitas 15y agoThe math puzzle we used required only prime numbers and modular arithmetic. Any competent programmer could wikipedia it and solve in 20-30 minutes (and many did).
- Jach 15y agoIt seems like this criticism should apply to FizzBuzz too, and that's why I'm inclined to dismiss it. (Specifically: how often does a Java business logic programmer invoke the modulus operator?) But I think I would agree with a more general point of avoiding difficult Project Euler style problems in favor of simpler ones that nevertheless need a written program to solve (or hardcore paper and pencil math skills).
- DanBC 15y agoFizzBuzz isn't about creating a correct compilable piece of software in a particular language. FizzBuzz is useful for finding the people who cannot even begin the problem; the people who couldn't craft even a vague approach in pseudocode. Some people can't sketch out a few loops and modulus tests. You know that the person who isn't sure of the precise syntax of a modulus operator can pick it up. You also know that the person who is baffled by FizzBuzz probably will need too much hand-holding.
- silentscope 15y agoI like this because it isn't a "gotcha" trick. That's the way this method backfires (in my opinion).
- kenrikm 15y agoThe only jobs that caught my eye were the ones that offered some type of challenge to apply (Apply Via API, Etc.) If I'm going to end up talking to someone from Human Resources that's going to ask me the "Man Hole Cover" or "Lightbulb Question" then forget about it, I'm not interested in working in more stuffy corporate environments. -(void)fizzbuzz { int i = 1;int t = 3; int f = 5;int h = 101; int z = 0; // here we do a for loop using i and incrementing it until it's equal to z for (i = i; i < h; i++) { /* we check if both i/t is equal to z and if i/f is equal to z if both are true we log fizzbuzz in the console */ if (i % t == z && i % f == z) { NSLog(@"FizzBuzz"); } //check if i/t = z and log else if (i % t == z) { NSLog(@"Fizz"); } //check if i/f = z and log else if (i % f == z) { NSLog(@"Buzz"); } // if none of them are true we just act like we're learning to count else { NSLog(@"%d", i); } } }
- Tyrannosaurs 15y agoDo you want a critique of that? I'm mainly interested because I'd be interested in a critique of my critique... We use Fizzbuzz as our lowest bar question and I'm curious to know what people consider significant points and what are not.
- waterlesscloud 15y agoSignificant point- Does it work? Non-significant points- everything else.
- Tyrannosaurs 15y agoDepends on what you're looking for. Does it work is the most important thing by a mile but if you've asked them to wrie production ready code then you've got to ask "how would I feel if one of my developers checked this into version control and how would it fair in a code review?" Maintainability, readability, efficiency and most significantly the candidates ability to explain why they took the approach they did. I wouldn't hold any approach against someone so long as they could talk intelligently about the trade offs and why they'd done what they'd done. Even really simple problems are a chance to see whether someone thinks about things.
- mattbee 15y agoYup, this worked really well for us too - we asked candidates to submit a 20-line "binary chop" search over a given data file (well, that was the solution, we described the problem and restrictions such that that was the only right answer). All the interviewees were good when we did that, and it also seems like a good test for recruitment agencies (i.e. which ones understand what we're trying to do). We did get one answer that was both incorrect, too slow, and massively over-engineered. I got emailed a multi-megabyte Rails project, boilerplate, tests for correctness and all. Bit It Didn't Solve The 20-line Problem. He was the guy we wrote the test to avoid!
- brandall10 15y agoThis reminds of the Google billboard test from years back: http://haacked.com/archive/2004/07/12/google-mysterious-billboard.aspx http://haacked.com/archive/2004/07/12/google-mysterious-bill...
- michaelochurch 15y agoI think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them. People who've taken Haskell or even Scala into prime-time production are extremely rare. People who would be excited to try, and who could be trained up into that role are a lot more common. I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.) I wrote about this effect here: http://michaelochurch.wordpress.com/2012/01/24/how-any-software-company-can-cross-the-developer-divide/ http://michaelochurch.wordpress.com/2012/01/24/how-any-softw... . It's long, but the TL;DR of it is that it's very easy for a company to become good at hiring if it gets its core technologies right.
- mmahemoff 15y ago"I think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them." A good way to do this is a humans.txt, whose purpose is not just to credit the developers but also the technologies in use. It's funny that a lot of companies are crazy about hiring right now, but haven't taken 30 minutes to add this simple file to their web server.
- scott_s 15y agoI'd go into management before I'd work in Java or C++ full-time. I suspect that statement is more absolute than you intend it to be. Would you not work on a new language if the compiler for that language generated C++, which meant that the language's runtime was implemented in C++? Which, of course, implies that you will have to write a decent amount of C++ code. If you would work on such a project, then I suspect you mean "If the problem and solution itself are not novel, then I have no interest in using Java and C++."
- michaelochurch 15y agoFair point. Actually, you've nailed it. C++, for example, isn't some horrible language without purpose. It has a niche that it fills well (although I'd prefer C) but most C++ jobs aren't in that niche. My experience is that language adjustment is up to a 0.5-point drop (the scale is here: http://michaelochurch.wordpress.com/2012/01/26/the-trajectory-of-a-software-engineer-and-where-it-all-goes-wrong/ http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) in an engineer's ability. This is on a 3.0 scale where most engineers are between 1.0 and 2.0, so that's a huge change. Compensation tends to be linear in proficiency but exponential in value-add. 0.5 points is about $50,000 in market pay but a 2-4x (depending on the type of company) difference in productivity. Sometimes it is worth the drop, such as when one's learning a dramatically new and more effective approach, or solving a fresh class of problem (e.g. real-time software, which is done in C). That isn't the case with enterprise, "pattern"-afflicted Java. Instead of solving the problem appropriately and well because I know how to do it, I have solve problems inappropriately and badly because I don't know the crappy idioms that people invented because they don't have appropriate tools. On that scale above, I put myself around a 1.7. If I set foot in a typical C++ or Java shop, I'd drop to 1.2 or 1.3. Which means I'd risk falling to the level of a junior programmer and being typecast that way. Which means I wouldn't get the benefit of the doubt if I say, "This project wouldn't be complete shit if we'd done it in an FP language" (in some nicer words, of course). Your first 6 months determine whether you have technical credibility or not. If you don't, you can't blame the tools, even if bad tools are (as they often are) the cause of the problem. The unenlightened tend to think of all tools as equally good (all languages are Turing complete). No, not so. A 3x difference in CPU performance may not matter for most problems, but the 3x difference in human performance that hinges on whether one uses the wrong tools certainly does. The drop to 1.2 is something I'd tolerate if moving to something genuinely interesting, knowing it'd only take a year to pop back up to the same or even higher level. An example might be high-performance C++ in weather forecasting, bioinformatics, or artificial intelligence. I'd gladly become a junior again for a year for that kind of stuff. On the other hand, most enterprise Java/C++ bullshit isn't really worth learning, not for a creature that has 60 years left to live instead of 600,000 (#ShitYouCantSayAtWork). So yes, my language bigotry is more directed at the typical C++ or Java job than at the language itself.
- michaelochurch 15y agoI think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them. People who've taken Haskell or even Scala into prime-time production are extremely rare. People who would be excited to try, and who could be trained up into that role are a lot more common. I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.) I wrote about this effect here: http://michaelochurch.wordpress.com/2012/01/24/how-any-software-company-can-cross-the-developer-divide/ http://michaelochurch.wordpress.com/2012/01/24/how-any-softw... . It's long, but the TL;DR of it is that it's very easy for a company to become good at hiring if it gets its core technologies right.
- guard-of-terra 15y agoThis reminds me of a stupid NH job posting where they said they want, and then there were two hashes: Googling for those hashes revealed in the first snippet first standing for "good", second for "developers". The message was that anybody who has the common sense to google an unknown string is qualified duh? The first puzzle job ad is nice, but it quickly wears off, and would start to bring bored "oh, not again" feelings. And many people just aren't good at making puzzles. It's the same as with paint-drawn NFS ad.
- csomar 15y agoIf you are looking for a good candidate, and ready to spend the time doing it, why not look around the developers who have done something in the field you are active on. For example, if you are looking for a Front-End guy, why not look around web magazines, Web 2.0 products and services, Github, Dribbble and pick the guys you like and contact them.
- SagelyGuru 15y agoGood puzzle, as opposed to a bad one, is an excellent attention getter for good programmers. They can judge you and your intellect by your puzzle and be attracted to work with you. Ie. your input makes the recruitment a two-way street.
- tokenadult 15y agoThe HUGE peer-reviewed professional literature on industrial and organizational psychology has many articles devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations. The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but still below the .50 level, standing alone). One is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Another is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Each of these kinds of tests has about the same validity in screening applicants for jobs. Neither is perfect (both operate at about .4x level in validation studies), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes. For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court http://scholar.google.com/scholar_case?case=8655598674229196978&q=Griggs+Duke+Power&hl=en&as_sdt=2,24 http://scholar.google.com/scholar_case?case=8655598674229196... held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Companies outside the United States are regulated by different laws.
- deleted 15y ago[deleted]
- 15y ago
- deleted 15y ago[deleted]
- rheide 15y ago"we hadn’t really planned for so many good applicants so we were very slow to respond to everyone" So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article. The key takeaway for me is not how people filter their applicants on real criteria, it's what they do when they have a large amount of seemingly equally skilled applicants. The original article demonstrated the 'unlucky' solution, I'm not sure what this article demonstrates since they still got too many applicants and they're spending time responding to all of them.
- SatvikBeri 15y agoHaving a lot of highly skilled applicants is a much better than having a lot of unskilled applicants. In the former situation, you pretty much can't go wrong...even if you use completely arbitrary filters, it doesn't really matter. This article demonstrates a way to get a lot of highly skilled applicants-that's extremely useful.
- yummyfajitas 15y agoSo they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article. The original article talks about finding 4-5 people worth talking to out of 100 people, 50 of whom can't fizzbuz. We were in the situation of 20 interesting people out of 40 candidates, all of whom could fizzbuz. If it were truly unmanageable, we'd probably have used weak Bayesian filters. E.g., degree, rockstar ninja, etc, all the things Raganwald originally advocated against using. I gather this is what Google and Facebook do.
- wheels 15y agoI call BS on this one. I'd even say that this will filter out the top level of folks (†). If you give me a synthetic puzzle to solve I wouldn't bother applying to your organization. It's much better to craft a problem to the specific domain that your company is dealing with. Then you can test for both domain expertise and problem solving ability in one fell swoop with the extra bonus of it not seeming pointless. † The important addendum on this is that you're not going to have the top level of folks sending out blind applications unless that person is in the process of making a lateral jump into another sub-field. And even then probably not. But if you give a top applicant a synthetic puzzle after your recruiting contacts them, expect a large portion of them to thumb their nose at you. Similar note on top folks, though this is mostly pedantry -- the top OSS projects aren't on GitHub and similar. The largest and most prestigious of the OSS projects host their repos themselves.
- kstenerud 15y ago"the top OSS projects aren't on GitHub and similar" True, however if you're a contributor to a top OSS project, you'd tell me that when you apply, and send me an example of one of your contributions that you're particularly proud of (at least I hope you do, otherwise I won't know!) Puzzles and challenges are useful when the candidate doesn't have any good examples of their work that they're at liberty to share with you. I'm very wary of hiring someone without seeing any of their work. We've been taking this approach for awhile now, and it's worked out well for us.
- yummyfajitas 15y agoTo clarify, I wasn't discussing recruiting Linus or Guido. I'm actually more likely to apply if there is a puzzle. I've sent a few resumes out blindly, it usually goes into the black hole. If there is a puzzle, I figure my odds of getting a response (assuming I get it right) go way up. Of course, if a correct solution results in a recruiter replying and asking me to submit to resumator, I'm probably ignoring your company in the future. (Yes, this happened to me once.)
- wheels 15y ago"Not trying to hire Linus" is a bit of a straw-man. There are a lot of folks that fall between "Linus" and "have to actively look for a job". Probably tens of thousands of people in the programming field. The important point that I'm getting at, however, is that the dynamics of hiring shift dramatically as you move up the skill / connectedness scale. As you move towards the higher percentages potential employees are not competing for employers, but employers are competing for hires. And the dynamics of hiring are dramatically different at that point. If you're designing heuristics for hiring, you don't want to create a low-pass filter, and I believe that synthetic puzzles are exactly that. The point of raganwald's post was on the surprisingly high cost of false-negatives. What I'm postulating is that your suggestion is just yet another way to reduce the false-positives rather than a way to reduce the false-negatives.
- doki_pen 15y agoPart of the trouble is that the best people are well employed and busy working. They are in high demand. They aren't going to do puzzles or beg you for a job. You have to find some way to meet them and convince them to work with you.
- somewhere1 15y agoIt is an interesting idea, but this may be limited in its use for small startups that hire only 1 or 2 programmers or for someone willing to wait. My reasons being: Only a small sliver of developers visit sites like HN. They are mostly busy working, going home to kids and such. Getting the news out that you are hiring, to this larger set of candidates is a real problem. Which is where the traditional methods (including job web sites) become crucial. If you use any job borads (like Monster), or a headhunter, it is hard to pipe a problem through them. They are not setup for this. If you had asked me to solve a problem just to find the email to apply to a job, I would have not done so, just because I do not have time to do it. If I were a kid out of college, or did not have time constraints, I would have loved to. But where I am today, I would rather send the resume in and talk to the hiring manager.
- lutorm 15y agoNo discussion of rockstar ninjas, just saying exactly what we are looking for. Like I posted in the "who's hiring", that doesn't work for me because it doesn't tell me whether you are what I'm looking for. What do you do? Why should I care about you?
- oconnore 15y agoAs someone who occasionally solves hiring problems with little intention of applying, it was nice to hear that my hobby is less annoying than I had imagined.
- edw519 15y agoHow to hire C players: Do what everyone else, including recruiters and HR departments, are doing. How to hire B players: Implement serious recruiting processes and plan to spend a lot of time on it. Get creative with hacks like OP's. How to hire A players: Identify them through never ending networking, research, and keeping your finger on the pulse of the industry. You'll never find them the same way you find B and C players.
- Tyrannosaurs 15y agoI think hiring A players is actually more complex than that even. To get A players, or even the best B players, the most important thing you can do is to make your company somewhere those people will want to work in terms of the work, the culture and the facilities. If you're doing that then it's possible they'll find you, if you're not then nothing you can do is going to help.
- Retric 15y agoIt greatly depends on what you call an A player. Linus is extremely talented and well known, but in some tiny organisations his actual programming talent may not be above average. Most programmers working for the NSA are probably about what you would expect at a large organization, but I suspect they have a few people that would rock your world. And the thing is Google, Microsoft, and GM also have some amazing teams simply though blind luck. The kind of thing where a few friends all end up working for a manager they like on some interesting little problem. Some people have only worked on such teams and don't understand why so many projects fail, and others have never even seen them and wonder how complex software ever get's written. But, so few people have a truly understand the range from inspiring though dysfunctional people and teams out there that talk of A, B, and C coders / rock stars becomes almost meaningless. PS: One of my favorite Real Programmers was a systems programmer for Texas Instruments. One day, he got a long distance call from a user whose system had crashed in the middle of saving some important work. Jim was able to repair the damage over the phone, getting the user to toggle in disk I/O instructions at the front panel, repairing system tables in hex, reading register contents back over the phone. http://www-personal.edfac.usyd.edu.au/staff/souters/Humour/Real.Programmer.Stories.html http://www-personal.edfac.usyd.edu.au/staff/souters/Humour/R... or the modern equivalent http://bellard.org/jslinux/tech.html http://bellard.org/jslinux/tech.html
- aangjie 15y agoHmm. I remember telephone interviewing for this position. As one of the applicants in the first group(i.e No puzzle just a simple job description posted :-)). I don't remember it verbatim, but the couple of questions i remember was (1. Why did you decide to apply?(i.e: from the post) I mentioned that the post referred to a entropy reduction algo and implementation. 2. Do you like HPMOR? :-P Just a reference. I don't have a strong opinion about the whole experience.
- jackfoxy 15y agoBefore I look at a resume I send every applicant a coding problem with simple instructions. It looks very simple at first, but actually requires some thought. It is fairly real-world, but unusual. I even provide a sample test case, and the instruction that the solution must pass all test cases. It would take the ideal candidate less than half an hour to code and test. Acceptable candidates less than an hour. Despite having all the time and resources they need to solve it, 85% to 93% of the candidates do not get it. I don't even glance at these resumes.
- gergles 15y agoHave you considered that 85-93% of the candidates don't want to do an hour's worth of unpaid work for you before you've even given them an interview?
- jackfoxy 15y agoIt's not 85-93% not responding, but 85-93% responding with crap that took some effort to produce. Also if you're actually interested in the position, you're going to be doing more than an hours work (if that's what you want to call it) in the interview process. Lastly, it would be work if I were tricking people into doing something I used as R&D or actual production end-product. This exercise simply establishes qualification.
- jarek 15y agoWhat is the response rate? How much contact do you have with the applicants before sending them the problem?
- jackfoxy 15y agoLately these are contract positions, and being in a corporate environment here I'm dealing with an agency that deals with other agencies. I still do some direct (usually by Craigslist ad) recruiting. I only submit the screening task to people who have already responded to the position posting. Very few people, maybe 5-7% do not complete the task, and I assume they found another position in the interim.
- cploonker 15y agoI found this very intriguing and decided to try. Here is my version of the filter: http://www.careesma.in/employer/joboffer/view/id/124996 http://www.careesma.in/employer/joboffer/view/id/124996 Any thoughts/feedback welcome.
- ralfd 15y agoWhat is this "Fizzbuzz" which only talented star programmers seem "able to do"?
- ben1040 15y agohttp://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html http://www.codinghorror.com/blog/2007/02/why-cant-programmer...
- SatvikBeri 15y agoFizzbuzz is a very simple problem described here: http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html http://www.codinghorror.com/blog/2007/02/why-cant-programmer... The point is not that everyone who can program FizzBuzz is talented. The point is that programmers who can't solve FizzBuzz are probably extremely untalented.
- joejohnson 15y agoWhat is fizzbuz?
- joejohnson 15y agoWhy are people so unhelpful? Is fizzbuz cryptography?
- aangjie 15y agoIt's an old blog post meme from jeff atwood http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html http://www.codinghorror.com/blog/2007/02/why-cant-programmer.... Simple googling for fizzbuzz programming finds it. EDIT: fixed wrong author name . thanks lazugod
- joejohnson 15y agoOh, you're right. Thanks.
- lazugod 15y agoAlthough that's Jeff Atwood, not Spolsky.
- mikeryan 15y agoI think this post misses one of the key takeaways from "unlucky". That post isn't only finding good candidates. Its about casting a wide net and doing the laborious part of filtering out good candidates solely on the basis of their qualifications. This seems like a good way to only get good candidates but you may also have some very qualified people who don't want to play this game and find it unnecessary. In some ways its the equivalent of throwing out resumes with misspellings.
- fowkswe 15y agoGreat way to keep recruiters out of the process too.
- shanemhansen 15y agoI used to think I knew something about hiring. I worked in a fairly large engineering organization and was involved in all our hiring. I did college recruiting, intern programs, phone screens and interviews. I mention that for background when I say: you can't reliably make good hiring decisions on the basis of a couple hours of your/the interviewees time. It just doesn't work. I'm personally a fan of hiring within your network (for engineering positions). You can't beat a genuine recommendation from someone who's spent months working shoulder-to-shoulder with the candidate. So please, when you hire people, have some humility. Know that your decisions are affecting someone's life in a major way, and you're probably making the wrong choice anyways, just do the best you can.
- ezl 15y agoi remember doing this. i mostly did the puzzle because it was a fun distraction. another thing is that it self selects for people who consider problem solving fun. its not necessarily the most predictive of job performance, but it is better than nothing and you'll also be filtering for personality (no opinion on whether that's a good or bad thing). sure it might weed out some good people, but it probably improves signal/noise ratio.
- Sottilde 15y agoI worked for a startup that used to advertise its open positions as "Silver Java Guru" or "Java Geek". Needless to say, we got one awful applicant after the other. Most could Fizzbuzz. Some could not. And others must have cheated their way through the phone interview - when you got them in person, it all fell apart. One thing I noticed from the process is that little things like spelling mistakes, incorrect formatting, etc., in a resume was very telling. Without fail, the kind of person who had typos in his resume was the kind that failed our programming tests, miserably.
- pgroves 15y agoSo with all the how to hire threads lately, may I make a suggestion: Focus on how good your company is on running an engineering department. Especially if you're a startup. Do you have a VP of engineering who knows what the hell they're doing? Do you insulate the devs from sales people making ridiculous promises? Is your code base in good shape so someone can come in and make a difference without having to learn the complete spaghetti stack? Do you have a well designed deployment procedure that has dev, test, and production environments? How do you approach scheduling? I'd say this is the number one issue that leads to bad experiences for developers, so tell us up front that you have this under control. Do I ever have to hear a business guy say "I don't see why that would take more than a few minutes..." In other words, prove that every problem isn't solved with "developers work more hours." Say something that at least implies you know that these issues exist. They are much more important than ping pong tables and calling each other ninjas. I bet you'll find plenty of pro developers who are sick of their current employer if you do. Imagine if developers actually learned something about running a software company just from your job ad itself. That would be amazing. I would point to JoelOnSoftware as proof it would work.
- ArekDymalski 15y agoI wonder when people stop searching for holy grail of simple, quick, universal and always effective hiring method. Even probation doesn't provide you with 100% accuracy of performance prediction. And everything else you'll use will be less effective. The real challenge is to make a proper, conscious choice of the tool taking into account the job itself (and its context) as well as balance between accuracy and resources spent. The discussions like this remind me discussions about which programming language is the best. Is there answer for such question? :)
- esonderegger 15y agoMy field (classical music) has a similar application screening problem (lots of applicants, very few good ones). Our chosen screening process, the audition, is imperfect for many of the same reasons given for why puzzle solving sometimes falls short when it comes to screening for developers: - The ability to perform on audition day doesn't necessarily correlate with the abilty to perform consistently in concert settings. - Great solo players aren't always great ensemble players. - It is very difficult to predict how an auditionee will mature over time. That said, our field has yet to come up with anything better, and I think the organization I work for has developed a particularly good screening process: On audition day, applicants are asked to perform some excerpts from literature our organization plays regularly, a prepared solo selection, and to do some sight-reading. This is pretty standard stuff for the clasical music industry and this preliminary round is done from behind a screen to help combat biases on the part of the panel. The excerpts are important because that's closest to the work applicants will be doing on a day to day basis if selected. The prepared solo helps show the candidates potential to excel in solo situations. The sight reading is crucial because it is a basic and necessary skill and correlates very highly with the applicant's quality of musical training. The second round (generally 5-15 applicants from the initial pool of about 100) is where our organization does things a little differently. It consists of more excerpts and sight-reading, but this time applicants are asked to make adjustments to how they performed the selections (a little faster or slower, a little louder or softer, etc.). Observing how the candidates respond to these requests shows how nimble the musicians will be in a rehearsal setting, and also shows how well-prepared they are and how well they will take direction. In the final round (generally 2-5 candidates), applicants are asked to perform alongside members of our organization. This demonstrates more responsiveness, how well the applicant listens, and how well the candidate performs in a group setting. With that in mind, I often wonder why those looking to hire developers don't use a multi-stage "audition" process. Step one could consist of some thing along the lines of "Here's a piece of code like what you might see in our organization. There's a bug in it. Find and fix the bug and send the revised code back to us." Step one could also consist of a short puzzle-type problem. Step one could be 100% automatic and would require the same amount of time for 100 applicants as it would for 1000 applicants. Step two could be where applicants are asked to show samples of something they've coded in the past. This could also be where they are asked to perform a more difficult bug fix and a more difficult puzzle. In this round, applicants are told that their code will be read for documentation quality and use of good style. Finally a final round could be done in-person where an applicant is asked to solve a problem with existing members of the team. This could be done in conjunction with one on one interview. This would help tell if the candidate is someone the team actually would enjoy working with, and how the candidate would be able to contribute. I know I wouldn't mind submitting to such an application process because I could see how each step is relevant to what the company is looking for. It also shows that the company takes hiring seriously. Hiring is supposed to be hard. Knowing that every one else on the team has been able to excel in those circumstances gives me greater confidence that I'd be joining a great team.
- deleted 15y ago[deleted]
- blhack 15y agoI fully support this mostly because I absolutely love doing the puzzles :). (So make more of them!)[1] One of my favorites from a few years ago was when reddit was hiring. One of the more fun aspects of it was: git clone our public repo and tell us the most commonly used word Another fun one was (I think? This was a few years ago): find me a right triangle, the length of which's sides all add up to: $some_number. [1]: In fact, this would be an awesome website if somebody hasn't made it yet: be a "recruiter", sortof, except you just post programming/math puzzles, we get to solve them, and then you get to match us up with employers.
- kami8845 15y agosee interviewstreet.com
- gruseom 15y agoWhat if companies just open-sourced their code? Then a programmer could look at the very system they would be paid to work on and a company could look at examples of real work. No more puzzles and arbitrary tasks. And if the system is of any interest and a community develops, that's a source of new programmers. The company gets access to a talent pool and hackers get to know if it's work they want to do. The obvious downside is that competitors get your code too. But is there any credible analysis or evidence of how bad a risk this is? Because the risk of not finding good programmers is pretty bad. I suspect that people are still way overestimating the value of source code in and of itself, relative to the value of the team that produces it. If so, that's a market inefficiency and open-source might provide an edge. It would also be more rational and satisfying from the point of view of the actual work. Imagine where mathematics would be if mathematicians still kept their proofs secret. Can anyone point to examples that shed light on the effect of open-source on hiring?
- manishjhawar 15y agoHow about putting a link to the real job post instead of "instructions on where to send your resume" hidden in the application?
- iamleppert 15y agoHave you ever stopped to realize that there is a class of people who like solving problems like these that have a definite, deterministic solution but who are utterly incapable of shipping code or a working product?
- nirvana 15y agoA, B and C are never really defined in these discussions, so I'm going to use my perspective: C programmers can't do FizzBuzz[0]. B programmers can, and LOVE PUZZLES. A developers are the ones whose idea of a "puzzle" is solving the fundamental business problem via software. B programmers think that super elegant engineering is great, and don't care about business so much. These B players are also the ones who often will torpedo a candidate by asking them trick questions (trick programming questions, irrelevant or obscure interview questions, etc.)[1] These questions are great for making the B player feel superior. Puzzles are a good way of doing this. This way if the candidate is hired, the B player's ego is mollified by knowing he was able to trick the candidate during the interview, and if the B player decides the candidate isn't deferential, he can claim he doesn't have the engineering chops for the job. I sometimes wonder how many of these complaints about people "not being able to do FizzBuzz" are from these B players. (I've never used FizzBuzz as an interview question, I don't ask programming questions, but I've never managed to hire someone who couldn't do FizzBuzz for a programming role.) At any rate, one of the things that makes interviewing at a given place a total roll of the dice is when you see evidence that they don't know what they're doing. An HR person trying to screen you who doesn't know the difference between java and javascript, for instance. But another signal to me, is these puzzles. These puzzles scream to me "This company is run by B players!" For instance, this is how I knew Google was not a company I wanted to interview with (even though they pursued me relentlessly for about 6 months, and still pester me periodically)... before I had the inside info that confirmed what the puzzles indicate. I also don't believe in the "A people hire A people and B people hire C people" mantra, because I worked at Amazon (A C organization if there ever was one) and Bezos liked to say this (a D manager, in my book... but obviously he thought he was an A.) I'm sure to some people, I'm a B player, or a C player (certainly the puzzle fanatics might think that) but I have seen a lot of companies get lost spending all their time working on clever engineering and not solving business problems. I like engineering problems and solving them, but the business problems are the really interesting ones-- its amazing what software can do to solve these problems, and often, non-engineering management has no way to know, which is why specs often don't work because thy ask for only what they think is possible. So, I'm not being a snob here. I'm just saying- I use these puzzles as a filter. I wouldn't apply to such a company (if I was still working for others.) [0] Theoretically, anyway. I'm a bit dubious about this claim-- all the programmers that I've seen who were a bad fit and "couldn't' do the work" were actually good programmers, just dealing with bad management. I have met a lot of non-programmers who think they are programmers and who only can do things that fit their pre-determined ways, e.g.: They're trained in a set of foundation classes and can't solve problems without using those classes (which have already solved things like sort, etc.) I assume these are the people we hear about when someone says "can't program fizz buzz". [1] For example, once I was interviewing for an internet infrastructure job and got quizzed on Win32 API details (like "what are the parameters for this obscure graphics call?" "I don't know, haven't looked at it in years." "It says here on your resume that you know Win32!". Similar example- interviewing for a Mac development position in the PowerPC era, mentioned I'd done assembly in Z80 back in the day and had a little familiarity with 8086 assembly-- getting questions about the Pentium instruction set. I could answer broad questions, because I had a high level exposure, but I only claimed to study 8086 back in the day, not be writing assembly currently, and it was for a Mac development job, and he wasn't even talking about the PowerPC.
- kylemaxwell 15y agoI suspect this works even better in areas outside of traditional development, like forensics, crypto, or data science. Hackers with specific skill sets, especially where those skill sets usually get applied to private data sets, often love the opportunity to mess around with alternate approaches on public data sets. And if it means an opportunity to show off, even better.
- ionforce 15y agoIf you're simultaneously someone who regularly reads Hacker News AND cannot FizzBuzz, shouldn't you pretty much kill yourself?
- ArekDymalski 15y agoNo, you should send yourself to the museum as an example of extremely rare specimen ;) Or learn yourself on codeacademy.com
- its_so_on 15y agoNote: the below may apply more to contractors, but if you read through it I think you might find it useful... in order of confidence, here is how you hire someone for the job. (perceived) 100.000% - The job is done already, just like something you already had the same guy do and it was perfect, now you pay him. 95% - The job is done to your satisfaction, you know the guy and he's solvent and vouches for what he's done. He hasn't done just this, but maybe something similar for you. You're not sure if this works as advertised yet... 90% - The job is done to your satisfaction, you know the guy and he vouches for what he's done. He hasn't done this for you but did it for someone else you know, who vouches for the guy. 85% - The job is done to your satisfaction, the guy comes recommended only by people you don't know, but warrantees what he's offering and seems like he cares about his reputation and about you, seems solvent and in a situation to fix it if it's not what you thought you were getting. 80% - The job is NOT done (none of it is), but you know the guy (rest per 100.000%) 75% - The job is NOT done (none of it is), but you know the guy (rest per 95%) 70% - the job is NOT done (none of it is), but you ... (rest per 90%) 65% - the job is NOT done (none of it is), but... (rest per 85%) 60% --> the job is not done and you have a guy offering to do it who comes recommended by people you don't know, you haven't worked with him, and he doesn't seem like he's in a situation to warrantee what he's going to do or care utmost about his reputation and about you. However, he brings proof he's done it before (portfolio). 55% - the guy comes with recommendations from people you don't know, but brings proof only of things OTHER than what you want him to do for you. He is not in a position to guarantee what he's going to do for you. 50% - the guy comes with recommendations from people you don't know, and doesn't bring proof of anything he's done. BINGO. This is where you are if you get a stellar resume and cover letter with references from people you don't know. Now of these resumes, let's go through the hiring filters that work... 40% of the time... 30% of the time... 20% of the time... 10% of the time... Ready? Okay, you know what, let's not go through all that. If you are trying to filter the hires so that you're getting the best of it like this, well, what can I say. Good luck pal.
- wilfra 15y agoThe lesson to be learned here is if you can do something different and get people to look at and talk about your posting, more people will apply because more people will see it. If the place all of those 'more' people see it is a place frequented by A players (such as HN), more A players will apply. In other words, this was a marketing/PR win - not a win in terms of how you are asking people to apply.
- dxbydt 15y agoLike it or not, this particular hiring filter seems to have caught on bigtime. The only downside to it is that there are people like myself who love to take a stab at solving puzzles regardless of whether they actually want the job or not. In the past 2 years, I solved online puzzles & scored interviews with the founders of 3 yc companies ( greplin, poundpay & freshplum) & 4 giant financial shops ( BlackRock & 2 other hedge funds I can't name/nda). In some cases like BlackRock, the puzzles are quite involved & can take a week to complete. Puzzles are a lot of fun....I just wish the associated job was equally interesting.
- K2h 15y agoDo you have a source for where you find these interview puzzles? I would like to take a few and see how they are set up.
- morenoh149 15y agointerviewstreet.com
- jisaacstone 15y agoThis is fairly true in my experience. I find solving puzzles at codeeval much more interesting than crafting custom cover letters, and the ease of applying means I'll more likely apply to a company there than elsewhere. OTOH I don't think I'd classify myself as an "A" player, at least not yet. The more I learn the more I find I need to learn &tc. (haven't heard back for anyone yet though. Probably a bunch of good applicants inundating their hiring team as well.)
- michael_nielsen 15y agoI've never been involved in hiring at a startup, but for 8 years I was involved in hiring as a prof in academia. In that context, there is a peculiar and consistent gap between the best hiring strategy and the strategies institutions actually use. For hiring faculty whose main focus is research, the best strategy is targeted attempts to hire superstars -- the research equivalent of a Guido van Rossum, say, or a Linus Torvalds, or someone near that level -- or early-career people where you have very good reason to be confident they will be future superstars. It sounds obvious to say "go after superstars". But what's strange is that in any given field only a very short list of institutions actually do it. At most academic institutions it's surprisingly difficult to convince the institution to go after superstars. I think the reason for this aversion is instinctive. If you're hiring it's much more comfortable to go the conventional route, advertising a position, interviewing a short-list and so on. This produces a situation where most of the interviewees want the job pretty badly. This gives the hiring committee a lot of power, which is a comfortable position to be in. Unfortunately, it also produces lower-quality candidates than identifying the best people and going after them directly. A superstar will be interviewing you more than you are interviewing them. You'll need to get creative in convincing them that you have something to offer, something they can't find anywhere else. This is much less comfortable. A useful heuristic: if most of your job offers are being accepted, your standards are way too low.
- stephan83 15y agoIt took me more than four months after college to find my first job as an intern. Six months later, I got offered a better job so I quit. However during the transition I was offered some well paid freelance gigs so I didn't end up taking the second job. After 4 years of freelancing and moving out of a big city, I got tired of working alone so I joined a small team in a non-tech company. Six month later, I was getting bored so I looked for work in a major city. I found one startup that did something that sounded interesting, so I sent them an email even though they weren't advertising opened positions. An hour later I got a response, and two weeks later I was working for them. The past 3 months, I got offered 2 jobs and Facebook contacted me for the second time. This isn't even the Silicon Valley -- I live in Europe. Sometimes I wonder if I'm smart or not. I know I probably wouldn't be able to answer all of these trick questions I hear about. I think I'm much better at abstracting concepts and seeing the whole picture. I love programming, but so far I haven't found a place where I can have as much fun as when I am creating software on my own, free of constraints and in control of everything. When I work on my own, every now and then I stop coding, sometimes for days, thinking, sketching, until I know exactly what I'm going to do and can explain why in plain English. Anyways, I think younger programmers might enjoy taking time answering these kinds of question, but I'm the kind of person that wishes he didn't have to sleep so he could spend more time working on his own projects, so I don't bother. If you want me to be interested, you'd have to find a way to convince me that I would be working in an environment that feels like I'm working on my own project. I want to work with motivated people that care more about their product than getting funded. I don't want to work in an environment where the number of closed tickets is a metric for productivity.
- prewett 15y agoIf I have to spend a lot of effort just figuring out if I even want to apply to the job/company, I'll probably just look somewhere else. On the other hand, if I'm not looking for a job but like puzzles, then maybe I might do it.
- gtani 15y agothis was good: http://lemire.me/blog/archives/2011/08/15/better-job-ads/ http://lemire.me/blog/archives/2011/08/15/better-job-ads/
- robbywashere 15y agoOMG I LUV SEEING PEOPLE JUMP THROUGH HOOPS TO EAT. HAHAHA LOL, L4M3rz amirite?
- jakobe 15y agoI wonder how many high quality applicants he would get if he just described what the fuck the job is about. I often look through job ads out of curiosity, and it's remarkable how generic most of them are. Maybe they promise "high profile clients" or "working with exciting technologies". Maybe they even promise "a friendly atmosphere". I don't understand why anybody would answer such a job ad. I'd be interested how many well qualified applicants they would get for a job ad that actually described what they are looking for: "We look for someone to work on the back end of our fizz buzz cloud service. The task will involve factoring numbers and counting digits."