8 ms·
Enough with the Programming Puzzles
- th0ma5 10y agoI think it is a very fair point that if you have to use a puzzle maybe you're not attracting the right candidates.
- collyw 10y agoYou are certainly putting off some candidates.
- wturner 10y agoThose puzzles might serve an ancillary goal of finding people willing to conform, are good at doing what they are told and have the ability to do unconventional programming tasks on a whim. All of those are characteristics that the company might be looking for. In short those tests aren't checking for just aptitude, they are checking for the personality characteristics of people willing to do them and tolerate the absurdity - like a cult. I once applied as a JavaScript developer to a local company and they gave me a test in Cake PHP. This was after a face-to-face interview with the owner. I finished most of it but their was one thing I couldn't figure out. His response via email was along the lines of " You really should learn this stuff". This was after answering a bunch of granular questions about JavaScript and passing those preliminary questions. But I guess its my fault and I "really should learn this stuff" (j/k) :/
- simula67 10y agoThese tests are probably designed to check your motivation, not your skill. If Google is asking questions about manhole covers, you better study about them before an interview. A candidate who can answer such a question would more likely want to badly work for Google. That is probably a better predictor for a good workplace performance than raw skill.
- TheOtherHobbes 10y agoIt's a good predictor for wanting to work at Google. It's in no way a good predictor for being able to do good work at Google - or anywhere else.
- ACow_Adonis 10y agoIsn't that what my university degrees were supposed to be for dammit! /ha ha only serious
- dclowd9901 10y agoIsn't it difficult to reject these puzzles on principle without sounding either bitter or like you simply can't solve them? I can't imagine a single case where this is going to cause a potential employer to re examine these exercises. They'll simply chock you up to being unqualified. I reckon most jobs simply require you to not be a knob for an hour while someone asks you tricky questions about your worst behaviors. How often, day to day, are you required to explain something bad about yourself in a constructive way? Yet it is as utterly common as programming puzzles. My supposition is that if that's where the wind is blowing, I'd had damn better face my sail that way. I hate silly problem solving puzzles as much as anyone, but not a shit lot you can do about it when they're the ones holding the keys.
- efaref 10y agoWell, I guess you could start your own company. You could even hire anyone who comes in the door claiming to be able to code, although what you'll do with a dozen people who can't even FizzBuzz is beyond me.
- vacri 10y agoSurely there's a middle-ground between puzzles designed to trip you up, and puzzles designed to show you have some baseline chops. There's a lot of people who can't walk their talk. At one place I work at, we're trying to get some senior-level PHP devs... and we're getting some applicants that can't even conceptually do FizzBuzz...
- collyw 10y agoIts not usually that difficult to work out if people know what they are talking about.
- efaref 10y agoSome people are really good at bullshitting their way through stuff. You need to make them actually do stuff to discover they flounder. And since you can't have them do anything meaningful in an interview (there isn't enough time), anything you ask them to do will be a silly "programming puzzle".
- asimpletune 10y agoI wonder if the author didn't know enough about the company to know if he wanted to do an online coding test, then how was he certain that the content wasn't relevant to what the company was looking for?
- lewisl9029 10y agoI'm actually in the middle of a bunch of interviews right now. The most ridiculous interview process I encountered so far was one that was supposed to be 7 (!!) steps long (that usually takes 3-4 weeks by their own estimates), beginning with a 45 minute HackerRank challenge, then a technical project that was said to take 2-5 hours on average (but the time given was a whole week, so I have my doubts about that estimate), followed by up to 4 Skype calls, and finally an onsite. The HackerRank challenge consisted of two relatively trivial multiple choice problems, a simple FizzBuzz-level coding problem, and a programming puzzle. And I do mean programming puzzle. I consider it a puzzle in the sense that it had very little to do with any classical CS domain I'm aware of (although that may just be an artifact of my own ignorance, but to someone unaware of that domain like myself, it as might as well just be a puzzle) and certainly wasn't anything one would ever encounter in front-end development (I was applying as a front-end engineer). I later found the problem online and here's a thread discussing it for those interested (the company I interviewed with wasn't Google though): http://stackoverflow.com/questions/28967020/check-if-there-exists-a-circle http://stackoverflow.com/questions/28967020/check-if-there-e... I finished the 2 MC questions and first coding problem within the first 5 minutes, but I wasn't able to finish the programming puzzle in the time remaining (For what it's worth, I was on the right track with my approach, but it took me a lot of trial and error to arrive at that approach, so I didn't have enough time left to finish the implementation). I got my rejection email two days later. I really question the wisdom of using a programming puzzle like this as a filter in the first step of your interview process (or really, in any step of the interview process for that matter). You're effectively discarding candidates solely based on their inability to solve an esoteric programming puzzle under some arbitrary time constraint. Although at the end of the day, I'm not that disappointed in this early rejection. If this programming puzzle was a sign of things to come in the next 6 steps of their interview process, I think I may have dodged a bullet. That could have been so many hours of my life that I'd never get back. Good riddance I say.
- penguinduck 10y agoYou dismiss this as "esoteric", but that's the whole point, the purpose of a puzzle like this is for you to be unable to map it to something you've done previously or to something you learned in school, and instead to force you to invent an algorithm on your own. It is the opposite of what the author of the article complains about - "academic", "targeted at cs grads" - it goes against that. It is an attempt to test problem-solving ability by removing knowledge from the equation. Knowledge can then be tested separately. Fresh CS grads are not going to do better on this problem than a guy who graduated 10 years ago. In fact, this is the kind of problem given to children (who can't be expected to have CS knowledge, and you don't want to give a big advantage to those who happen to have some) in programming competitions. I would put the difficulty at either easy problem for high schoolers or difficult problem for elementary schoolers (this doesn't mean it's super-easy, these problems have to be relatively hard or the smartest kids would all solve everything). I am not sure why so many developers dismiss the value of this and are in fact often actively hostile to any suggestion that it might be ONE of the indicators of programming ability. I do agree that the time constraint was probably too strict and favors people who've solved stuff like this before, mainly due to them knowing they should try to solve this with a pen and paper or in their head, then just code the solution once they've found it. Trial and error through code is a huge waste of time and this was probably your problem.
- kinkdr 10y agoUs, software engineers, start to sound like spoiled kids when it comes to interviewing. We don't like writing code on the whiteboard, we don't like mind teasers, we don't like programming puzzles, and so on. We complain about everything. Well, hiring is a difficult problem, and nobody has a universal solution yet. But the truth is that if I want to get a job, I will need eat the humble pie and get out there and try to perform as good as I can to whatever I am asked. If I don't, somebody else will. However, remember that interview is a two way process, if I don't like the employer's process, then maybe I will not even interview for them. In this case though, I cannot expect them to give me an offer, just because.
- pandaman 10y agoIt seems that other professions have solved this. Nobody is asking lawyers to run a trial on a whiteboard or solve a whodunit mystery in 2 hours. And I don't think that licensing process for lawyers guarantees anything other than the right to practice law (there are pretty bad licensed lawyers out there). Do construction workers draw diagrams of installing drywall in their hiring process? Do plumbers get challenged with the back of cereal box trace-the-pipe puzzle? There are jobs, which have a similar hiring process. Performance artists actually need to perform to get hired but they do perform their actual work. I.e. they are not playing the anthem of Cambodia on a toy recorder in order to be hired as a death metal band drummer.
- sametmax 10y agoIn those other jobs get hired a lot : 1 - by social networking; 2 - on diploma. For 1, The problem is, programmers will be recruited either by: - non programmers. And dev will likely not be found in the same social circles than those people; - programmers. Which tend to use a technical solution to all problems. For 2, we all know now that you got as many good and bad dev, no matter the diploma, so people have given up. And yes, dev are spoiled in the US, so recruiting technics move fast, trying to adapt. But in France, they are not, and we have the same old recruiting technics. And it's not better. Lot of bad dev are recruited.
- 10y ago
- WatchDog 10y agoI don't know what the solution to hiring is, but I don't think it is as simple as just talking to people. While I have not been involved in too many new hires, I have known many people that come across as knowledgeable in conversation but produce pretty terrible code. I think coding challenges provide some value, but it can be a but much to expect candidates to spend hour's of their time early in the process
- m0nty 10y agoDo people applying for other roles get treated to tests like this? Seems peculiar to IT afaict. Having worked in IT for > 25 years, I would have thought my track record would speak for itself, but still I'm treated as though I've been lying and cheating my way through my career. It's not like just asking questions, either: it's a test of basic aptitude and (like the author) I'm at the stage of refusing to do them.
- kriro 10y agoThere's assessment centers for most decent entry level management jobs which usually involve grouping tasks by importance and team oriented tasks. They can last a day, sometimes two. Typical consulting gigs have case studies and market size guesstimating (the dreaded how many lawnmowers in NYC etc.) Typical office secretary jobs here require a work sample so you'll get a typical task for the job and have to do it on the spot (write a letter to someone, translate a text). My current job (academia) required a somewhat simple test to see if I have a decent understanding of method/statistics and the programming test was answering a couple of questions :P In logistics you usually have to reason through a couple of typical work situations. After entry level...it's usually just an interview and references.
- m0nty 10y agoI guess that's encouraging, kind of. I've always felt like this whole thing was just an extension of "don't trust the IT guy", a viewpoint a one-time employer was happy to articulate loud and often ... in front of the IT guys. I once had an agent almost freak out over the phone because he'd seen my open-source project and couldn't believe I'd created it myself. "It had to come from somewhere", I told him. "Why shouldn't it be me as much as any other random stranger on the phone?" Seems like even when I presented them with impressive, real-life work, they still didn't want to trust me :)
- gaius 10y agoAfter entry level...it's usually just an interview and references. Except it's not, that's the problem. But this is not a matter of incompetence on the part of the employer, but of deliberate and calculated policy.
- divkakwani 10y agoI think a candidate needs to have both algorithmic and software design skills. Companies put a lot of emphasis on the former skill, ignoring the latter, which imo should equally be tested.
- jarcane 10y agoIf you want to find out if a programmer can actually do the job, then it seems to me that the way forward is to have them do something that actually resembles the job. In the high-end cooking world, it's not uncommon to do one of two things when looking to hire a new cook: 1) Make me a dish. Present the chef with the kitchen, and possibly a specific main ingredient(s), and have them make the best dish they can with that ingredient. Sometimes this is also under some kind of time constraint, to test pressure under fire, but doesn't have to be (great chefs might take weeks to develop a new recipe after all). 2) The "stage". The kitchen takes on the chef temporarily for a short period, where they will actually work on different portions of the line, so that the employer can see both how they perform in real time and where their strengths lie. This can be as little as a single shift, or up to a month or more at some very, very high end kitchens like Thomas Keller's. Crucially as well, this is generally paid (it'd actually be illegal otherwise), so the prospective hiree can actually afford to dedicate that kind of time. Now some of this is similar to stuff that some tech companies do already. But I think they often miss crucial elements that makes this actually work in a way that's not painful for both parties. The "make me a dish" method for instance, superficially resembles the infamous "sit down at our computer and code a thing" tests I've read of many times. But there are key differences. Tooling is a pain point in programming that doesn't so much exist in cooking (in fact part of this test can be making sure the prospective cook is familiar with all the basic tools). Sitting in at a foreign editor/IDE/language etc. is likely to be more stressful than is really accurate to how one can expect a new coder to perform in the real world. The simple solution is "bring your own knives." Let the coder use their own machine, or at least give them time to set up a basic version of their favorite tools. The other thing about "make me a dish" that's missing from programming equivalents that is I think most important is creativity. "Make me a dish" isn't about just proving you can cook, it's about proving you can create. And isn't that something we want from great programmers as well? Generally when I see and read about these tasks their very proscribed: there's a specific spec/demand/puzzle, and often with the expectation it will be solved in a particular way. One test task I was given once actually even specified what data format I was supposed to use to communicate between back and front ends. What about a mini-hackathon/jam approach instead? Give me a subject or a data source and a bit of time, and see what I can come with? The "stage" as well is something I think is underconsidered with tech companies. Companies spend thousands of dollars on hiring practices and recruiters and advertising, but they can't just do a paid trial period instead? It makes no sense. If you think you've got a promising candidate, just take them on for a couple weeks, point them at a relatively low-hanging fruit task on the issues board, and see how they do. Alternately, maybe just accept a fucking risk now and again. If the candidate seems to know what they're talking about in the interview, maybe just hire the fucking person, and if they suck, fire them. This is the same stricture and risk 90% of hiring practices are under, and somehow the economy hasn't collapsed yet so I am gonna tentatively suggest it seems to work. Sometimes the path of least resistance is the easier, cheaper way.
- gaius 10y agoTypical LinkedIn "thought leader", didn't Google et al abandon brainteasers over a year ago?
- joshvm 10y agoThe article isn't talking about brain teasers. Google did indeed banish them, but now they've switched to 4-5 interviews where you're asked a short coding problem. The problems themselves aren't difficult and given the time constraint, you can fit the answer in the height of one whiteboard. The trick is that you have to be fast, optimal and correct (i.e. syntax counts). From my experience, you should be able to complete each problem in about 20 minutes - i.e. a good enough solution that the interviewer moves on. The article is right: there are books and books on doing these toy problems with array manipulation, tree traversals, sort/search and so on. If you did a TopCoder SRM every night for a few months, you would probably ace most of these interviews. But this is the issue: the interviews basically test how well you prepared. They don't simulate a work environment and I don't think I was ever asked any serious questions about my work experience/research beyond chit-chat. Given that I was applying for a specialist computer vision role, that was a bit surprising.
- laichzeit0 10y agoDo you really need to test whether a candidate is technically competent if they have a degree from a well respected university like MIT, etc? You already know he has the discipline and academic requisites. Interviewing someone at that point, I'd expect the employer would be more interested to see if they're a good fit for the company culture and not waste time on academic trivialities.
- greggman 10y agoWell, Laszlo Bock claims Google looked at the data and found zero correlation between having a degree from a well respected university and being useful at Google http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman-how-to-get-a-job-at-google.html http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman... So no, using a degree isn't useful for hiring engineers apparently
- laichzeit0 10y agoThat's pretty weird though. Is using a degree useful for hiring medical doctors, mechanical engineers, actuaries? It's really hard to believe that only Computer Science as a field has a problem where it's churning out people who are technically incapable of programming to such a degree that you need to explicitly test them in interviews through "puzzles".
- greggman 10y agoMaybe they all have that problem and no one has ever bother to actually check the data? I know for me and interviewing programmers my personal experience is the degree doesn't matter. I've met plenty of degreed "engineers" who couldn't program for shit from schools like MIT and Harvard. I've also met some amazing programmers from those same places. I don't know what make the difference except possible love of the activity. Not sure how to asses that in an interview or whether it correlates better with outcomes.
- icebraining 10y agoFor medical doctors, having a degree just means you get to spend years in a residency training program under supervision. SEs also have internships, but they usually want someone they don't need to train from 'scratch'.
- hartror 10y agoAs an interviewer I am upfront with our hiring process and that part of it is we ask candidates to do a "exercise" after the first face to face meeting. This exercise is crafted to be representative of the job we are trying to fill and is followed up by a second face to face code review session of the code. The key of the exercise isn't necessarily to complete the whole task (though often people do as the exercises are interesting) but to provide a discussion piece for the code review session. Candidates are told this so that they don't feel like they have to put too much work into it (2 hours is suggested). The code review session is key, walking through someones code and discussing decisions and trade offs. It provides a layered insight into their smarts, communication skills and thought processes. It is the closest I've gotten to actually working with someone in the interview process. And isn't that what both parties are looking for?
- philbarr 10y agoOne of the problems with coding challenges is that I suspect you miss out on a lot of potential candidates. I did a round of interviews recently and one wanted me to do an online test and then a puzzle. I did the online test, received the puzzle, and in the intervening period I was interviewed and offered a job by someone else. When I rang the first place to tell them the guy was very upset because it had taken so long to find someone who could pass the test! Seems to me if you're that desperate for candidates then don't assume you're the only company in the world.
- pmyjavec 10y agoI would have to largely agree with this based on my own experience, and I've been on both sides of the fence. The best interviews I've had was when I was invited to come work with the team after a few meet and greets. It allows all parties to relax a little and get a taste of what working together would be like.
- sandworm101 10y agoSo I guess we know the answer to the "How would you react if assigned a task you didn't enjoy?" question. A: Rather than spend two hours getting the job done, an hour is spent doing a write-up on why the task wasn't done. A job, any job, will involve thousands of hours with at least a few hundred of them on tasks you really do not appreciate. A candidate not willing to suffer for a couple hours during the interview probably wouldn't have lasted very long anyway.
- philbarr 10y ago> A job, any job, will involve thousands of hours with at least a few hundred of them on tasks you really do not appreciate. But you get paid to do those tasks. And remember that those 2 hours are an estimate, which means it'll probably take longer. Then multiply that by the number of interviews you have and you can quickly see why it becomes unfair rather than 'lazy'.
- efaref 10y agoSometimes you have to work on spec.
- collyw 10y agoWell done for replying back with a reasoned response for not participating. I have done this at a few places. I hope if enough people do it people will get rid of these interview techniques.
- HemanHeartYou 10y ago>Instead, I sent this email to the recruiter: >If y'all