19 ms·

Take-home vs. whiteboard coding: The problem is bad interviews

146 points by gwbas1c 7y ago

the_arun 7y ago
Even if you do - openly ask them for a feedback and keep improving it. Showing empathy & keeping your ears open always helps.
- gwbas1c 7y ago
  OP here, I agree, good idea!
ldoughty 7y ago
Live coding (open internet) had been one of the great tools to help evaluate people. We don't want programmers that necessarily know our programming language already, but you should know how to search for map/filter/sort built in functions. We also leverage take home assignments, which helps people get brushed up before the live challenge... It also tells us where people are in their skills -- code commented? It self-commenting? Any error handing? Etc. I've also pushed for this take home work to be relatively short, it's unfair to have candidates spend 4+ hours in a project with no promise if an interview. It always driver me nuts when businesses asked for that, so I try to avoid it myself. All this said, it had dramatically United the quality of our candidates we actually see in an interview, which saves our developers time interviewing someone who can't figure out how to do JavaScript sorting using built in functions, or printing out only of numbers in an array of alphanumeric values.
renewiltord 7y ago
We have a take-home and a whiteboarding bit. No tricks to the whiteboarding. Very straightforward brute-force solution that you can improve. Take-home doesn't have code. It's a technical design question that you then discuss with an engineer on an on-site. So far so good. I sometimes attempt to requalify by adding a second step where I try an alternative mechanism of testing, just to see if someone who did poorly on the whiteboard (which is a coderpad-style thing) would do better in a different mechanism. So far very high correlation. Pity, I'm very social so I'd prefer a conversation and walking through code at a higher level.
keb_ 7y ago
My favorite interview process was with Wikimedia. It was a take-home coding project, which I then had to discuss with multiple engineers on the team over the course of four interviews (it was a remote position). I didn't get the job, but it was very enjoyable being able to work on something without someone watching me, and then being able to discuss my design decisions with different engineers who had different opinions about how I could improve my work. It was clearly valuable regardless if I got the position. When presented with white-boarding interviews, I'll usually feel physically sick. This includes intense sweating and weirdly enough, gassy-ness. I found this old blogpost pretty relatable: https://aneccodeal.blogspot.com/2014/02/interviewing-for-anxious-programmer.html https://aneccodeal.blogspot.com/2014/02/interviewing-for-anx...
- mLuby 7y ago
  Agreed. I prefer testing how well a candidate can dive into other peoples' code, understand it, and make changes to meet a spec. To respect their at-home and in-person time, that means sending them a small and straightforward codebase, making them extend it in clearly pass-fail ways, and then when they come in you have them talk about the code and extend it a bit more. The take-home portion serves a few purposes: it filters out uninterested applicants, weeds out non-coders, and lets people prepare in their own time and way (rather than on the spot in an interview). The in-person portion then checks that the candidate can do the work themselves, can communicate about their work, isn't obviously a pain to work with. Most take-homes test the wrong things: whether someone can start a greenfield project, think up interesting new features, or put in more extra hours than their competition. And most in-person interviews test the wrong things too: have you studied interview problems, are you good at coding and problem-solving out loud under time pressure, and how well do you scrawl syntactically correct code on a whiteboard.
  - ProZsolt 7y ago
    I have only one thing to add. Don't make it a pre-screen test. I want to know before I do any take-home test that the job is actually a good fit for me.
- fapjacks 7y ago
itsmefaz 7y ago
I might get lot of flak for this, but everytime I see a post criticising the software engineering interview process I'm thinking what is this dude trying to sell. The interview process is an assessment on your programming knowledge and its also an assessment on whether you'd be able to hang mentally (intellectually) around the people within the organization (read culture). If the culture of the organization is made of people who are very proficient in computer science theory. It is imperative they'd would expect any new individual to at least be like them. That's human nature!
- gwbas1c 7y ago
  That's exactly what I'm trying to sell! You make my point better than me! I'm also not criticising the process... The process is good! "The problem is bad interviews" is meant to direct criticism to bad interviews. A religious war of whiteboard vs take-home vs whatever interview format is in fashion is silly.
jerzyt 7y ago
I completely don't understand interviewers who like to haze the candidates. When I interview a candidate, I'm rooting for him or her. My employer has already invested time and money in the candidate, I have invested the time to study the resume and prepare relevant questions. My interviews are pretty challenging for the candidates, but I try to make the atmosphere friendly.
- gumby 7y ago
  They don't care and feel unappreciated. So here's an opportunity to feel better. Who cares if the company needs someone? Sadly I've even seen this from a company founder who felt that as the company grew they were no longer the "big kahuna" and felt a need to make up for it by showing off in an interview. (experiences as a co-interviewer not candidate)
  - eru 7y ago
    They should channel their showing off into displaying how awesome and sociable and good with candidates they are..
    - gumby 7y ago
      if only. A nd they should be glad to be able to work with such smart and solid folks so everything doesn't rest on their shoulders. The most fun organizations are where everyone is glad to be working with all those other people and things everyone else is smarter than them (so with a lot of opportunities to learn). Organizations like that really do exist -- I've worked in a bunch, but the larger the organization gets the harder that is to maintain. But when a founder is doing this it poisons the well right away.
- coffeefirst 7y ago
  Because they're not interviewing people, they're playing the role of interviewer that they once saw on TV. The goal of the interview is not to subject someone to a test, it's to become informed about them and their abilities to do the job. The industry has this weird defensive position where everyone believes there's an army of imposters trying to invade them, and it's become a complete parody of itself.
vkaku 7y ago
I've said this before, but here I go again: The point of an interview is to bring the best out in a Candidate, not to conform that the interviewer knows better, and definitely not to feed u to their ego.
jordansmithnz 7y ago
From being on both sides of whiteboard and take home interviews over the last few years, I’m all for whiteboard interviews, for a few reasons: - take home interviews incentivize the candidate to spend more time than they’re told. You can choose a three hour problem, but candidates that really want the job will probably spend more time. Less experienced candidates will spend more time on it too. - I never found reviewing take home interviews to be a great indicator of skill level. It’s hard to know how long was really spent on it, and how much the candidate used resources etc to guide them. Are they a great candidate that cut corners, or a less experienced candidate that used stackoverflow + 10 extra hours to get to a final result. - whiteboarding can go badly too. The caveat is that if it does, only so much of the candidate’s time is taken up; there’s a hard ceiling. - from the perspective of conducting whiteboard interviews, the company loses out if the interviewer doesn’t do a good job. This is really on the company though - it’s worth investing in a good whiteboarding process. If there’s no investment here, it’s only the company that suffers (competent candidates will get hired by a company with a better process). In terms of conducting a good whiteboard interview, my $0.02 is no trick questions, just solid problem solving with plenty of scope to show creativity and proficiency.
- colordrops 7y ago
  Take home interviews should be time-limited and proctored real-time. If it's more than 4-6 hours then it's too long. The problem should have multiple parts and just difficult enough to make it unlikely for most candidates to complete everything. The problem should unique enough and broad enough to allow the candidate to demonstrate good code structure, problem solving, creativity, and algorithmic ability. It should obviously not be something so straightforward and simple as to be a question on stack overflow. The candidate should be encouraged to submit early and often for feedback, and to not be afraid to ask questions. Sometimes a subtle or complex element of the problem statement can help spur this interaction. Finally the grading should be a blend of both objective criteria such as a series of heuristics and unit tests, as well as subjective measures such as code cleanliness and structure. Multiple people should be involved in the grading, and a repository of previous submissions should be compared against for consistency. I and my team used this technique for years to great success, hiring some of the world's top talent.
sytelus 7y ago
CS whiteboarding makes sense when tech stack isn't going to be constant and the tasks you might be doing requires more than making calls to backend from MVC. On the other hand, take-home stack specific question makes sense otherwise. The bottom line is that software engineering jobs are very different but we keep expecting to put on same cloths on them in terms of interview process.
- clarry 7y ago
  > CS whiteboarding makes sense when tech stack isn't going to be constant and the tasks you might be doing requires more than making calls to backend from MVC. I'm not inclined to agree. If I look at my job, well, it's rather "full-stack", spanning from hardware to bootloader to kernel to init system to application level. Debugging techniques range from probing with an oscilloscope, disassembly, dissecting core dumps, printf-tracing of asynchronous multithreaded systems, packet captures and traffic analysis at raw ethernet level.. I can't think of a single "CS whiteboarding" question that would do much to help me select a candidate for my position. As far as algo wizardry goes.. dude, naive O(n) and O(n*n) algorithms work for damn well near everything, and if you can figure out how to use one of the existing binary tree libraries or call bsearch(), you're pretty much set. I'm sure we could give you a few days to figure out some more advanced algorithm if profiling proves it to be necessary for some particular job. Honestly I think the most important skill is the ability to learn new stuff and dig deep. I don't know how to test for that in an interview, I really have no clue. Probably understanding concurrency and races is one of the more "critical skills" and I'd like to see that in a candidate, but I don't know if lacking a good grasp of that is grounds for rejecting a candidate. All too often I see lists prerequisite "critical skills" that any good programmer would be able to pick up on the job as needed. Do you really need to test people for something they can figure out in a day or three and get good at over the coming few weeks?
  - sytelus 7y ago
    Again, you are assuming all software engineering jobs are like yours. I’d been on projects where I’d to deeply think about graphs and literally invent variations of algorithms that worked for my scenario. I’d to written code for tree searching, online median finding, distributed minimum tree spanning etc in my various projects. If you are working in things search, AR, DL related products anyone will tell you that 1% of improvement in algo performance often translates to millions of dollars of savings. Unfortunately designing and evaluating algorithms is very hard earned skills and candidates who have internalized tools of this trade do much better than regular full stack folks. CS whiteboarding is precisely for such candidates but unfortunately full stack guys also started adopting them for their jobs. For full stack projects that need little to no algorithms design, take home coding question specific to tech stack works much better.
logicslave 7y ago
Algorithms interviews are IQ tests, the end.
- DeathArrow 7y ago
  This isn't necessarily a bad thing. But it's not only an IQ test: it also assesses how good is someone at solving problems. The individual shouldn't come with the perfect answer in a limited amount of time but show a good thinking process which put him on the right track to solve the problem. When doing coding someone can have to solve an issue which doesn't have an answer on stackoverflow, so having problem solving skills is valuable. Also, being good at algorithmics and data structures enable someone to come up with better solutions which might matter in a particular case. I think algorithmic questions are better than asking questions about random methods from a particular framework.
  - pdr2020 7y ago
    It also assesses how good someone is at preparing to solve specific kinds of problems. Which is fine. If you're naturally talented at whiteboarding, great. If not, but you've prepared months ahead of time, great also. If you're not naturally talented, and haven't prepared because of naivety or due to some principle, well, that's not exactly a great signal to an employer.
- oreally 7y ago
  More like Algorithms interviews are a way for software engineers to stroke their own ego, the end. If software engineers actually took the time to look at how intelligence is actually measured in fields outside their own, and how it relates to job performance, we'd have a lot less broken interviews.
- zerr 7y ago
  No, it's actually a quite opposite - trivia test from the field of Competitive Programming.
jimbob45 7y ago
Look, if you spend five years with a girl, you’ve gotta marry her. Clearly, you didn’t find anything wrong with her enough to break up and she can’t wait around forever. Are there better women out there? Almost definitely. They didn’t just commit five years to your chubby ass though. Likewise, once you make someone put 4+ hours into tech work and it’s good work, you’d better hire them or apologize profusely for wasting their time.
- michaelborromeo 7y ago
  You should be looking at one woman over those five years. For companies, you are probably looking at multiple developers and if you do only have one opening then even if more than one is good there is going to be a best.
- joe_the_user 7y ago
  The company is only going hire one person. If they make ten people do their task, they've committed themselves to wasting the time and energy of all at least nine of those people, even if all ten people do something amazing with that assignment.
vlttnv 7y ago
> You Can Ask the Candidate to Present Code they Wrote. I have always wondered why this isn't more common, it makes so much sense to me. When you submit an application you usually submit links to your blog, GitHub, projects etc. When I attend an interview I expect us to talk about those things. This is what the person is supposed to be good at. Instead you are met with a predefined set of puzzle questions. I think the purpose of the interview is to allow the candidate to show what they are good at and what they are excited about and to see if this fits well with what your company is doing.
- repiret 7y ago
  When I was young and single, I programmed in my free time. Now I have a wife and three kids and a house and I like woodworking and flying and fixing up old buildings. I sell all of my programming output to my employer, who doesn't happen to make open source. In my free time, I play with my kids or fix something on a house or build something in my shop or something that doesn't leave a mark on GitHub. If you are evaluating me for a job and ask me to show you some code I've written recently, I can't do it. I do want you to test me, because I want to work in a shop where everybody can pull their weight - I've worked in shops that don't test candidates, and they end up with some really bad programmers who are sometimes decent sweet talkers. But give me something on a whiteboard, or a take home thing that I can finish in a couple hours after the kids go to bed, or just let me bang on a keyboard in your offices or whatever. If you ask your candidates to show you some code they wrote, you're selecting for candidates who either do open source in their day job, or don't have hobbies that don't produce code. There's lots of great programmers that meet neither of those criteria.
  - vlttnv 7y ago
    Fair point! I guess if I can edit my statement a bit I'd phrase it like that: If from my application you can see that I have work that we can talk about and discuss, let's do that, if not then let's do the predefined questions. That's fine. I've had a couple of situations where I go in an interview and the interviews ask me questions that show to me that they have not even looked at my CV and the cover letter they asked me to write. They were just reading off a list of generic tech interview questions. To me that feels like if I went to interview at a company and told them I have no idea who they are or what they do, I just know their name and that they are looking for a coder. That's just counterproductive. But I totally agree with you that there is no single solution. I just think there are different "good" questions for different people, interviewers and interviewees need to adapt to that.
dahart 7y ago
One of my favorite interviews ever was squarely in-between whiteboarding and take-home. I was put in a room by myself with access to the internet and a computer and asked to write a specific game in any language I wanted. The rules of the game were given, but everything else was up to me. The main requirements were to be able to make a move, and to calculate the current score. For extra credit, make an AI player. There were other phases of the interview, coding wasn’t the whole thing, but I really enjoyed it. I bombed the object modeling on a whiteboard part of the interview, but the company made a great offer anyway based on being able to code. I’m modeling my hiring now on this type of interview because it was both fun and effective.
TrackerFF 7y ago
I actually enjoy take-home exams more, because a good whiteboard interview absolutely relies on having a good / competent interviewer. Take-home exams obviously also need competent people to analyze the results, and also have their pitfalls - especially if they're just being used as free labor.
seanmcdirmid 7y ago
I opted for an easy take home from Twitter recently when applying for a job there. However, the problem was so horribly written and vague that I couldn't figure out what they really wanted, even after asking questions. Honestly, I would have been better doing a problem with time constraints interactively, at least I could get some decent feedback about what was expected. Flubbing the Twitter take home worked out for the best, Google was a much better fit for me anyways. Their interview process was great also, and doing whiteboarding over VC affords certain advantages (getting to type in code, if only into Google docs, is a huge win for me, Jamboard + an iPad Pro worked out great as well).
- anonymoushn 7y ago
  When I interviewed at Google last year, they said they offered every candidate a Chromebook on which to type code instead of using a whiteboard, but the office I visited had no working interviewee Chromebooks. Maybe most people who go to on-sites can type code on a real computer.
baron816 7y ago
I think it would be interesting to ask a candidate what question they normally ask when they conduct interviews, and have them to walk through the problem. Has anyone here tried that or heard of that being done? I think you could potentially learn a lot more from that than you could with your own problem. First, there would be a lot less stress on them since it would be a problem they're presumably very familiar with. If they do a terrible job coding it up, well, that's a super clear signal. Second, stuff like how well the do setting up the problem, what kind of problem they choose (is it some generic Leetcode problem, or something they came up with themselves), what they look for in a good solution (just that it works/that it's over-optimized/obeys some arbitrary style standard/etc.) would all provide a less clear signal, but still useful information, especially regarding seniority.
- LanceH 7y ago
  That's such a loaded question, though. It is completely open ended. Will they be judged on the difficulty of the question? Their ability to answer it? Is it even emphasizing the right area for this interview? I strongly believe the candidate should be given every possible direction on what they are being judged on. I'll ask, "What are some of the things you look for in a code review?" If they hesitate or seem unsure about how to answer, I'll help them out with "What are some of the things that will get a PR rejected?" From there I try to open it up to a conversation rather than getting them to list the "correct" five things to cover in a code review.
- pmiller2 7y ago
  I couldn’t technically answer that question. My company uses a structured interview process, and all our questions fall under NDA.
srich36 7y ago
New graduate here who has this admittedly been on only one side of the interview process. In my interviews I got mostly white boarding problems and one take home. The author was mostly correct in discussing the advantages and disadvantages of both (namely the extra-time for take home assignments and archaic algorithms for white boarding) but I feel there are a couple points I can add. First, I actually believe white boarding questions (at least the automated kind) are assigned more indiscriminately than take home assignments. Many companies just send you online timed hackerrank or leetcode questions as a preliminary filter. Second, the entire interview process rewards people who study for the interviews rather than those who like to build projects. Unfortunately I believe companies miss out on a lot of qualified candidates, albeit reducing their false-positives from obscure algorithm-type problems. With so many applicants, reducing false-positives ensures quality hires so it works on the company’s end. Overall, my opinion is the interview process is relatively broken. Take-home assignments offer more insight into actual development skill, while whiteboard questions filter out a large portion of people (many who would be just as good, if not better, at actual implementations.) But it works for the interviewers so change seems unlikely. This is just my experience for new graduate interviews; perhaps it is different at higher levels.
- ddrt 7y ago
  It could reward those people because it exemplifies following a spec and completing to spec. The whole thing, studying and showing you will do things you don’t want to do (and do them well), and complete a really absurd set of requirements that have seemingly no value. However, you will stand out if you ask “why did you choose those questions?” Or “how did you come up with those problems?” Even the best interview that fits in all the check boxes trumps a similar candidate who shows they give a shit.
- pdr2020 7y ago
  Wouldn't this be fixed by presenting a on-site assignments? For example a small project, with test suites, where a feature or two needs to be completed and tests updated? (Disclosure: We did this at a company I worked at).
- izacus 7y ago
jillesvangurp 7y ago
I've been asked to do coding interviews for freelance gigs a couple of times recently. This does not make sense to me. The whole point of freelancing is that either of us can terminate the contract on short notice if for whatever reason we don't like each other. Usually, if it comes up, I just recommend the company that if they have another qualified candidate (i.e. better qualified than me), they should do their best to hire them. It's a good way to cut the bullshit and get it in people's heads that they need to work hard to get me if they want me to commit to their project. If based on my profile they are not super eager to talk to me, I'm wasting my time because either they need somebody with a different profile or they are B's trying to hire C's. In both cases, that's a good reason for me to not engage any further. I've been in the interviewer role as well and I tend to not rely on coding exercises, whiteboard BS, etc. I usually just wing it knowing full well that interviews are in any case a bit of a crap shoot in terms of objective results. My goals when interviewing are very simple: 1) verify that what is in the person's CV holds up and try to get them to talk about what they did, how they did it, etc. 2) inject a few technical questions to see if they can come up with some good answers or whether they start bullshitting. 3) Figure out what they don't know and how they would deal with having to learn something new. 4) Figure out how well their experience aligns with my team's needs. I'm OK with a less than perfect match if a person has the potential to learn new stuff. Especially with junior roles, that's the most important thing because they probably don't know much yet. But really, I tend to go with my first impressions here and have learned the hard way that ignoring those leads to bad hires. Yes, there's a risk I'm wrong about not hiring somebody. But if it feels wrong, it's not going to happen. Usually, I know within a few minutes what it's going to be; the rest of the interview is then either about making sure the candidate is sold on the job or giving me enough material to defend the choice not to hire to my managers, HR, etc. Coding exercises don't really factor into this. If I have any doubts whatsoever about a person's ability to actually produce code, I'm not going to hire them. Unfortunately hiring is hard. There are a lot of mediocre candidates out there with a lot of mediocre recruiters representing them. I vastly prefer warm introductions through trusted friends and colleagues. So, as an interviewer, I deal with screening a lot of obviously not great candidates. The flip side is that when somebody good comes along, you need to be able to act fast because they'll have other options. Coding exercises are an obstacle for both sides.
atoav 7y ago
A technique that I learned from casting actors is: you don't have to be the candidates enemy to find out if they fit. If you ever find yourself trying to actively make the candidates life harder for no reason, you are doing it wrong (e.g. asking trick questions, expecting weirdly specific answers, etc) Instead give them multiple different hard (but realistic for the position) problems and be on their side every time to guide them. Try to get them to do the best they can and observe them all the time. This way you can get good feeling for how they are as a person, how they react to guidance and teamwork, and false negatives where a simple miscommunication leads to wrong solutions and a bad result, despite the candidate actually beeing good, are reduced. If they constantly and heavily rely on your guidance that is a bad sign. If they actively fight against your guidance, this is bad as well. But in the end you are ”casting” them for a certain position within an existing "cast", and sometimes a person who initially needs more guidance will do great after a while if they are in an environment where they can receive it, while people who actively avoid guidance could work just fine in a position where they don't get too much anyways etc. The opposite of this approach would be to ask an actor which pieces of [insert obscure european postramatic theatre director here] they know and if they know none assuming they can't act. Bonus points if the project doesn't have any connection to the topic asked at all. If anybody ever does this, it says more about the person asking, than the person answering. No — try really hard to get the best out of the candidates in the most accurate emulation of the job you want to give to them, and have a grasp of what they will get up to speed with in no time and what actually will get worse (e.g. personality traits). And if you areninsecure about someone give it another shot in a second round.
- pmiller2 7y ago
  I think you’ve got it right. The best interviews I’ve participated in felt like working with someone for an hour or so. As for the opposite scenario, you’ve got that right as well, except that the interviewer is a 25 year old, recently graduated theater major.
  - atoav 7y ago
    > As for the opposite scenario, you’ve got that right as well, except that the interviewer is a 25 year old, recently graduated theater major. Insert there what you want, hehe. It always reminds me of one teacher I had, who always wanted to get oddly specific answers on his (always a little to unspecific) questions. He was so happy when nobody could answer it, he probably forgot what his job actually was. Behaviour like this tells more about the psyche of the interviewer than it tells about the candidate. Sometimes a behaviour like this is also the interviewers way of making sure the candidate doesn't expose them as the fraud they deep down feel they are: "See they didn't know that oddly specific bit (that I know!), so they must be worse than me – thank god!" In case of my teacher he felt insecure enough to feel the need to one-up teenagers and instead of teaching them something. Interviewers are only human after all as well. But it shouldn't be the candidates job to rub interviewers in the right way in order to make them comfortable with their own shortcomings as it shouldn't be the pupils job to massage teachers into actually teaching them something.
godot 7y ago
I really enjoyed this article. It's probably one of the most fair, unbiased, and insightful articles written on tech interviews and may be the best one I've read. Over the past 15 years, I have interviewed with many different companies. I once read that when you're a good fit for the company, the interview will feel suspiciously easy. This has been the case for all the times I've gotten offers. Reading this article, a lot of the good points remind me of a particular interview I had though. It was at relatively large tier-2 tech firm a few years ago (not the FAANG tier); one where I didn't get an offer, but the interview process and experience was memorably good. I remember specifically telling the recruiter even after the rejection that I really enjoyed the interview there. One of the rounds was with the principal engineer at the company, who was the most senior engineer at the company at the time if I recall correctly. It was the system architecture interview. I was asked to prepare by thinking of a system I had designed and architected in my past experience, and present it as if the principal engineer was my coworker and we would hold tech discussions. It's the closest thing I've ever gotten to the "Present code they wrote" idea that Andrew mentions here. For the coding round of this interview, I was asked to bring my own laptop (another point that Andrew mentioned here; this was also the only company of all the interviews I've had in 15 years that asked me to do this.). I was then given a problem at the beginning of that interview, and given 45 minutes or so to write code on my own (the interviewer actually walked out of the room and left me alone while I did). It was a reasonable problem to code for the allotted time, and had reasonable complexity but not overly difficult. When time's up, the interview came back and we discussed my code. Both of those rounds were something that was so well done that I think it was the most fair way a company could have assessed me correctly. I did well on the coding round, but my experience in system architecture was not a good match for the type of work they did, and in retrospect I could tell. Since I was interviewing at the Staff Engineer level, I think they correctly evaluated me and turned me down for the role. (Although I would point out that after that day of interview, they ended up being undecided, and asked me to come back one more time to have one more interview round, with one of the first engineers of the company there. I did poorly in that round for a variety of reasons, and that was ultimately what got me the rejection.)
youeseh 7y ago
Whiteboard interviews all the way. They are interactive. And they can be two-way interviews. All participants of the interview are interviewing each other. Take homes are basically free labor.
toolslive 7y ago
Why don't they trust the universities? If the candidate has a degree in computer science shouldn't he be able to code ?
- OutThisLife 7y ago
  Not in my experience.
  - pkolaczk 7y ago
    The same applies to algorithmic challenges given for whiteboard coding. The fact one can reverse a linked list doesn't mean one would be a good developer in a real world project. Whiteboard coding doesn't check the basic system design skills.
    - dragonwriter 7y ago
      Whiteboard coding isn't designed to find good all-round developers, it's designed to filter the candidate pool to decrease the likelihood of candidates being selected that don't have the necessary coding chops. Other aspects of the selection process check other aspects of the desired skill set.
- repiret 7y ago
  While I think that ought to be a requirement for getting a CS degree, not all universities do. Moreover, being able to code isn't a boolean - while some people are indeed totally incompetent, among the competent people, some are better than others, and some have strengths and weaknesses that would make them more ore less suited to a particular role. It behooves an employer to try to measure these things.
- kortilla 7y ago
  Computer science doesn’t teach you how to write code.
- rsynnott 7y ago
  ... No.
lifeisstillgood 7y ago
Is hiring broken for non-software jobs like chemists, physicists? What about writers or journalists? I only have vague insights into these industries but science jobs afaik rely a lot on recommendations - on calling prior collegues and asking. And for writers - well reading their previous stuff works well (easier with open source stuff) So i would suggest right approach is more a simple FizzBuzz screen, a cultural fit (careful of biases!) and then reading their past OSS / other work (so maybe a prepared portfolio is a good "take home") and calling their references.
- ProZsolt 7y ago
  then reading their past OSS / other work A lot of people don't have any polished open source code. I have a lot of stuff on Github, but I wrote it for myself. I don't care if it brakes If I only using it once or twice a year.
deleted 7y ago
[deleted]
topkai22 7y ago
As an interviewee, I have a particular distaste for take home questions, at least if given before doing a couple phone screens- I want to see a company invest time in me a bit before spending half a day coding. Besides, having been an interviewer more than an interviewee at this point I find being able to competently discuss previous projects with a degree of excitement more predictive of success anyway. I still ask to see interview coding, but it's more a check box for basic skills than a major focus
stared 7y ago
While the example was bad (too general, a potential for copy&paste from somewhere else), it was certainly not for the reason OP wrote: > The exercise was supposed to take "a few hours" according to the hiring manager, but I'm extremely rusty with ASP. Well, if they look for an ASP expert, this self-selection saves time on both sides! I use take-home assignments with boilerplate code, to reduce the initial time investment and make it easier to compare parts of the projects. (Otherwise, it may be a too open-ended task.) I try to make sure that the skills I require the skills I need. For example, when I needed a deep learning engineer (for a PyTorch project), I expect fluency in at least one deep learning framework. I made it explicit that instead PyTorch boilerplate then can use any other framework (a few people send in Keras). Yes, people can Google things (good!). And it some sense it balances more junior people (but willing to put more effort) with people with more experience. The later could be told easily by interview. When I ask questions about their solutions, it is easy to see which parts are done with understanding, which - not.
- gwbas1c 7y ago
  OP here. The reason why the stack requirement bothered me was that, during the phone screening, I basically told the hiring manager that take-homes are fine, but I walk away from anything that has a laundry list of required frameworks. They were looking for a very senior developer, not someone who would come in and know their stack on day 1. In my case, the problem wasn't ASP, it was all the other frameworks and patterns they wanted.
lmilcin 7y ago
From the point of view of interviewer. I find that take-home exercises have two major flaws: 1. The push the effort onto interviewee. Interviewee should not be punished for the interviewer's inability to prepare good questions and exercises and inability to design the process so that bad candidates are filtered out early in the process. 2. They are not really meaningful. I've been there as a candidate. It is difficult to voluntarily stick to the timelines given when you know you could do this or do that to improve the solution and that other candidates will not hesitate to do it. As an interviewer, I have no idea if the candidate did the exercise by themselves or with outside help. Some (not all) candidates will do anything to game the system. You don't want to get these people onboard. Based on that I am strongly against take home tasks. On the other hand I like whiteboard exercises because they let me discuss things in an environment and closely to circumstances we could be having at work. It is normal for me to draw algorithms or even implement pieces of (pseudo) code on the whiteboard and to use it for discussion. What I try to keep in mind that different people deal differently with the whiteboard. Some people like to visualize, some think a little bit more abstract. I had to work hard to be able to draw anything in a way that will be digestible by other people and I still think am doing only mediocre job of it.
mrfusion 7y ago
I’ve always wondered how mechanical or civil engineers are interviewed? How about accountants? You never hear about interview problems for them. As an industry we should take our dignity back.
- benstrumental 7y ago
  Those professions usually require a license gated by a standardized exam, which ensures some level of competency before showing up to the interview.
bitwize 7y ago
Back when I used to do whiteboard interviews, the trick was not to grade the candidate on right or wrong answers, but to look for signs of engineering thinking. The problem was simple, but well-suited to low-level languages like C, and we would even give them hints if they forgot some aspect of C-family syntax. BSers would reveal themselves in obvious ways, and anyone who displayed engineering competence was still in consideration.
tcrow 7y ago
We have a big app and need people who can come in and get up to speed very quickly. A solution to the interview problem that I've adopted over the last few years, to great success, is to conduct a 2 hour, 2 phase interview. - The first hour is spent having the candidates work through a problem in a sand boxed environment using the actual code base. - The second hour is spent recapping the work session, discussing their professional background, and general questions about other things they should know to be successful on our team. The coding test is designed to be solvable within the allotted time and without explicit knowledge of the business rules in the app and only requires an understanding of the actual technology they purport to have the required experience in. Running this test is a great way for us to understand what the candidate's skill level is and what problem solving approaches they use in a real-world scenario. They are free to talk about what they are doing (but is not required) and of course we offer guidance if they get stuck on something. Solving the test is NOT a pre-requisite for getting hired. In order to reduce anxiety for the candidate, I like to frame the test by telling them to imagine themselves as a short-order contractor who is coming in to help us with a problem that we are having a hard time solving ourselves. I find that this really helps. The feedback from the candidates for this testing style has always been positive, and I can tell you from post-hire experience, that it has helped us to bring some great developers on board. So far, there have been no false positives. In fact, it's so popular that other teams have either asked me to conduct their interviews or plan on adopting the same methodology for conducting theirs. If your struggling to get more consistency from your hiring approach, it might be worth a try.
geebee 7y ago
One thing I prefer about whiteboard coding interview exams is that the do require an investment of time from the interviewing company. Take-homes allow them to externalize the horrendous inefficiency of the interviewing process. Beyond that, I think articles like this are important to read and circulate widely. Young people need to know that if they become doctors, lawyers, nurses, actuaries, licensed processional engineers, dentists, or financial planners, they will indeed need to take a test. However, the test will be managed by a central authority, with predictable content, with a clear study path, and once you take the entrance exam, you won't have to take it over and over during job interviews for the rest of your life. There may be refresher exams, but again, they will be managed by the licensing body and will provide a clear preparation path, and will be graded consistently. It won't all be left to the whims of whoever thought up a question. If you become a dev, you will deal with interview exams, often conducted by people who are not really qualified to conduct exams, every time you interview, even if you have decades of experience. If you are a senior actuary, you will not need to integrate by parts every time you interview, but if you are a developer, you may very well have to find all matching subtrees in a binary tree every time you interview. You are signing up for a lifetime of retaking your intro to algorithms midterm. You may be able to avoid this by starting a consulting company, or going into management, but at this point, if you plan to be a "knowledge worker", I would highly recommend taking a look at other fields.
samsquire 7y ago
I feel that we (techies) make hiring miserable by asking for the perfect versions of ourselves. The result is entry level inflationism.
SergeAx 7y ago
When I'm giving candidate a take-home assignment - I'm always asking for time and effort estimation, both "net" and "gross", like in "how many hours will it take" and "when it will be ready". Actually, ability to give viable estimation and stick to it is more important than coding itself. As for other matters in article, my current company just paying candidate for finished assignment, plain and simple.

146 points by gwbas1c 7y ago

Even if you do - openly ask them for a feedback and keep improving it. Showing empathy & keeping your ears open always helps.

gwbas1c 7y ago
OP here, I agree, good idea!

ldoughty 7y ago

Live coding (open internet) had been one of the great tools to help evaluate people. We don't want programmers that necessarily know our programming language already, but you should know how to search for map/filter/sort built in functions. We also leverage take home assignments, which helps people get brushed up before the live challenge... It also tells us where people are in their skills -- code commented? It self-commenting? Any error handing? Etc. I've also pushed for this take home work to be relatively short, it's unfair to have candidates spend 4+ hours in a project with no promise if an interview. It always driver me nuts when businesses asked for that, so I try to avoid it myself. All this said, it had dramatically United the quality of our candidates we actually see in an interview, which saves our developers time interviewing someone who can't figure out how to do JavaScript sorting using built in functions, or printing out only of numbers in an array of alphanumeric values.

renewiltord 7y ago

We have a take-home and a whiteboarding bit. No tricks to the whiteboarding. Very straightforward brute-force solution that you can improve. Take-home doesn't have code. It's a technical design question that you then discuss with an engineer on an on-site. So far so good. I sometimes attempt to requalify by adding a second step where I try an alternative mechanism of testing, just to see if someone who did poorly on the whiteboard (which is a coderpad-style thing) would do better in a different mechanism. So far very high correlation. Pity, I'm very social so I'd prefer a conversation and walking through code at a higher level.

keb_ 7y ago

My favorite interview process was with Wikimedia. It was a take-home coding project, which I then had to discuss with multiple engineers on the team over the course of four interviews (it was a remote position). I didn't get the job, but it was very enjoyable being able to work on something without someone watching me, and then being able to discuss my design decisions with different engineers who had different opinions about how I could improve my work. It was clearly valuable regardless if I got the position. When presented with white-boarding interviews, I'll usually feel physically sick. This includes intense sweating and weirdly enough, gassy-ness. I found this old blogpost pretty relatable: https://aneccodeal.blogspot.com/2014/02/interviewing-for-anxious-programmer.html https://aneccodeal.blogspot.com/2014/02/interviewing-for-anx...

mLuby 7y ago
Agreed. I prefer testing how well a candidate can dive into other peoples' code, understand it, and make changes to meet a spec. To respect their at-home and in-person time, that means sending them a small and straightforward codebase, making them extend it in clearly pass-fail ways, and then when they come in you have them talk about the code and extend it a bit more. The take-home portion serves a few purposes: it filters out uninterested applicants, weeds out non-coders, and lets people prepare in their own time and way (rather than on the spot in an interview). The in-person portion then checks that the candidate can do the work themselves, can communicate about their work, isn't obviously a pain to work with. Most take-homes test the wrong things: whether someone can start a greenfield project, think up interesting new features, or put in more extra hours than their competition. And most in-person interviews test the wrong things too: have you studied interview problems, are you good at coding and problem-solving out loud under time pressure, and how well do you scrawl syntactically correct code on a whiteboard.
- ProZsolt 7y ago
  I have only one thing to add. Don't make it a pre-screen test. I want to know before I do any take-home test that the job is actually a good fit for me.
fapjacks 7y ago

itsmefaz 7y ago

I might get lot of flak for this, but everytime I see a post criticising the software engineering interview process I'm thinking what is this dude trying to sell. The interview process is an assessment on your programming knowledge and its also an assessment on whether you'd be able to hang mentally (intellectually) around the people within the organization (read culture). If the culture of the organization is made of people who are very proficient in computer science theory. It is imperative they'd would expect any new individual to at least be like them. That's human nature!

gwbas1c 7y ago
That's exactly what I'm trying to sell! You make my point better than me! I'm also not criticising the process... The process is good! "The problem is bad interviews" is meant to direct criticism to bad interviews. A religious war of whiteboard vs take-home vs whatever interview format is in fashion is silly.

jerzyt 7y ago

I completely don't understand interviewers who like to haze the candidates. When I interview a candidate, I'm rooting for him or her. My employer has already invested time and money in the candidate, I have invested the time to study the resume and prepare relevant questions. My interviews are pretty challenging for the candidates, but I try to make the atmosphere friendly.

gumby 7y ago
They don't care and feel unappreciated. So here's an opportunity to feel better. Who cares if the company needs someone? Sadly I've even seen this from a company founder who felt that as the company grew they were no longer the "big kahuna" and felt a need to make up for it by showing off in an interview. (experiences as a co-interviewer not candidate)
- eru 7y ago
  They should channel their showing off into displaying how awesome and sociable and good with candidates they are..
  - gumby 7y ago
    if only. A nd they should be glad to be able to work with such smart and solid folks so everything doesn't rest on their shoulders. The most fun organizations are where everyone is glad to be working with all those other people and things everyone else is smarter than them (so with a lot of opportunities to learn). Organizations like that really do exist -- I've worked in a bunch, but the larger the organization gets the harder that is to maintain. But when a founder is doing this it poisons the well right away.
coffeefirst 7y ago
Because they're not interviewing people, they're playing the role of interviewer that they once saw on TV. The goal of the interview is not to subject someone to a test, it's to become informed about them and their abilities to do the job. The industry has this weird defensive position where everyone believes there's an army of imposters trying to invade them, and it's become a complete parody of itself.

vkaku 7y ago

I've said this before, but here I go again: The point of an interview is to bring the best out in a Candidate, not to conform that the interviewer knows better, and definitely not to feed u to their ego.

jordansmithnz 7y ago

From being on both sides of whiteboard and take home interviews over the last few years, I’m all for whiteboard interviews, for a few reasons: - take home interviews incentivize the candidate to spend more time than they’re told. You can choose a three hour problem, but candidates that really want the job will probably spend more time. Less experienced candidates will spend more time on it too. - I never found reviewing take home interviews to be a great indicator of skill level. It’s hard to know how long was really spent on it, and how much the candidate used resources etc to guide them. Are they a great candidate that cut corners, or a less experienced candidate that used stackoverflow + 10 extra hours to get to a final result. - whiteboarding can go badly too. The caveat is that if it does, only so much of the candidate’s time is taken up; there’s a hard ceiling. - from the perspective of conducting whiteboard interviews, the company loses out if the interviewer doesn’t do a good job. This is really on the company though - it’s worth investing in a good whiteboarding process. If there’s no investment here, it’s only the company that suffers (competent candidates will get hired by a company with a better process). In terms of conducting a good whiteboard interview, my $0.02 is no trick questions, just solid problem solving with plenty of scope to show creativity and proficiency.

colordrops 7y ago
Take home interviews should be time-limited and proctored real-time. If it's more than 4-6 hours then it's too long. The problem should have multiple parts and just difficult enough to make it unlikely for most candidates to complete everything. The problem should unique enough and broad enough to allow the candidate to demonstrate good code structure, problem solving, creativity, and algorithmic ability. It should obviously not be something so straightforward and simple as to be a question on stack overflow. The candidate should be encouraged to submit early and often for feedback, and to not be afraid to ask questions. Sometimes a subtle or complex element of the problem statement can help spur this interaction. Finally the grading should be a blend of both objective criteria such as a series of heuristics and unit tests, as well as subjective measures such as code cleanliness and structure. Multiple people should be involved in the grading, and a repository of previous submissions should be compared against for consistency. I and my team used this technique for years to great success, hiring some of the world's top talent.

sytelus 7y ago

CS whiteboarding makes sense when tech stack isn't going to be constant and the tasks you might be doing requires more than making calls to backend from MVC. On the other hand, take-home stack specific question makes sense otherwise. The bottom line is that software engineering jobs are very different but we keep expecting to put on same cloths on them in terms of interview process.

clarry 7y ago
> CS whiteboarding makes sense when tech stack isn't going to be constant and the tasks you might be doing requires more than making calls to backend from MVC. I'm not inclined to agree. If I look at my job, well, it's rather "full-stack", spanning from hardware to bootloader to kernel to init system to application level. Debugging techniques range from probing with an oscilloscope, disassembly, dissecting core dumps, printf-tracing of asynchronous multithreaded systems, packet captures and traffic analysis at raw ethernet level.. I can't think of a single "CS whiteboarding" question that would do much to help me select a candidate for my position. As far as algo wizardry goes.. dude, naive O(n) and O(n*n) algorithms work for damn well near everything, and if you can figure out how to use one of the existing binary tree libraries or call bsearch(), you're pretty much set. I'm sure we could give you a few days to figure out some more advanced algorithm if profiling proves it to be necessary for some particular job. Honestly I think the most important skill is the ability to learn new stuff and dig deep. I don't know how to test for that in an interview, I really have no clue. Probably understanding concurrency and races is one of the more "critical skills" and I'd like to see that in a candidate, but I don't know if lacking a good grasp of that is grounds for rejecting a candidate. All too often I see lists prerequisite "critical skills" that any good programmer would be able to pick up on the job as needed. Do you really need to test people for something they can figure out in a day or three and get good at over the coming few weeks?
- sytelus 7y ago
  Again, you are assuming all software engineering jobs are like yours. I’d been on projects where I’d to deeply think about graphs and literally invent variations of algorithms that worked for my scenario. I’d to written code for tree searching, online median finding, distributed minimum tree spanning etc in my various projects. If you are working in things search, AR, DL related products anyone will tell you that 1% of improvement in algo performance often translates to millions of dollars of savings. Unfortunately designing and evaluating algorithms is very hard earned skills and candidates who have internalized tools of this trade do much better than regular full stack folks. CS whiteboarding is precisely for such candidates but unfortunately full stack guys also started adopting them for their jobs. For full stack projects that need little to no algorithms design, take home coding question specific to tech stack works much better.

logicslave 7y ago

Algorithms interviews are IQ tests, the end.

DeathArrow 7y ago
This isn't necessarily a bad thing. But it's not only an IQ test: it also assesses how good is someone at solving problems. The individual shouldn't come with the perfect answer in a limited amount of time but show a good thinking process which put him on the right track to solve the problem. When doing coding someone can have to solve an issue which doesn't have an answer on stackoverflow, so having problem solving skills is valuable. Also, being good at algorithmics and data structures enable someone to come up with better solutions which might matter in a particular case. I think algorithmic questions are better than asking questions about random methods from a particular framework.
- pdr2020 7y ago
  It also assesses how good someone is at preparing to solve specific kinds of problems. Which is fine. If you're naturally talented at whiteboarding, great. If not, but you've prepared months ahead of time, great also. If you're not naturally talented, and haven't prepared because of naivety or due to some principle, well, that's not exactly a great signal to an employer.
oreally 7y ago
More like Algorithms interviews are a way for software engineers to stroke their own ego, the end. If software engineers actually took the time to look at how intelligence is actually measured in fields outside their own, and how it relates to job performance, we'd have a lot less broken interviews.
zerr 7y ago
No, it's actually a quite opposite - trivia test from the field of Competitive Programming.

jimbob45 7y ago

Look, if you spend five years with a girl, you’ve gotta marry her. Clearly, you didn’t find anything wrong with her enough to break up and she can’t wait around forever. Are there better women out there? Almost definitely. They didn’t just commit five years to your chubby ass though. Likewise, once you make someone put 4+ hours into tech work and it’s good work, you’d better hire them or apologize profusely for wasting their time.

michaelborromeo 7y ago
You should be looking at one woman over those five years. For companies, you are probably looking at multiple developers and if you do only have one opening then even if more than one is good there is going to be a best.
joe_the_user 7y ago
The company is only going hire one person. If they make ten people do their task, they've committed themselves to wasting the time and energy of all at least nine of those people, even if all ten people do something amazing with that assignment.

vlttnv 7y ago

> You Can Ask the Candidate to Present Code they Wrote. I have always wondered why this isn't more common, it makes so much sense to me. When you submit an application you usually submit links to your blog, GitHub, projects etc. When I attend an interview I expect us to talk about those things. This is what the person is supposed to be good at. Instead you are met with a predefined set of puzzle questions. I think the purpose of the interview is to allow the candidate to show what they are good at and what they are excited about and to see if this fits well with what your company is doing.

repiret 7y ago
When I was young and single, I programmed in my free time. Now I have a wife and three kids and a house and I like woodworking and flying and fixing up old buildings. I sell all of my programming output to my employer, who doesn't happen to make open source. In my free time, I play with my kids or fix something on a house or build something in my shop or something that doesn't leave a mark on GitHub. If you are evaluating me for a job and ask me to show you some code I've written recently, I can't do it. I do want you to test me, because I want to work in a shop where everybody can pull their weight - I've worked in shops that don't test candidates, and they end up with some really bad programmers who are sometimes decent sweet talkers. But give me something on a whiteboard, or a take home thing that I can finish in a couple hours after the kids go to bed, or just let me bang on a keyboard in your offices or whatever. If you ask your candidates to show you some code they wrote, you're selecting for candidates who either do open source in their day job, or don't have hobbies that don't produce code. There's lots of great programmers that meet neither of those criteria.
- vlttnv 7y ago
  Fair point! I guess if I can edit my statement a bit I'd phrase it like that: If from my application you can see that I have work that we can talk about and discuss, let's do that, if not then let's do the predefined questions. That's fine. I've had a couple of situations where I go in an interview and the interviews ask me questions that show to me that they have not even looked at my CV and the cover letter they asked me to write. They were just reading off a list of generic tech interview questions. To me that feels like if I went to interview at a company and told them I have no idea who they are or what they do, I just know their name and that they are looking for a coder. That's just counterproductive. But I totally agree with you that there is no single solution. I just think there are different "good" questions for different people, interviewers and interviewees need to adapt to that.

dahart 7y ago

One of my favorite interviews ever was squarely in-between whiteboarding and take-home. I was put in a room by myself with access to the internet and a computer and asked to write a specific game in any language I wanted. The rules of the game were given, but everything else was up to me. The main requirements were to be able to make a move, and to calculate the current score. For extra credit, make an AI player. There were other phases of the interview, coding wasn’t the whole thing, but I really enjoyed it. I bombed the object modeling on a whiteboard part of the interview, but the company made a great offer anyway based on being able to code. I’m modeling my hiring now on this type of interview because it was both fun and effective.

TrackerFF 7y ago

I actually enjoy take-home exams more, because a good whiteboard interview absolutely relies on having a good / competent interviewer. Take-home exams obviously also need competent people to analyze the results, and also have their pitfalls - especially if they're just being used as free labor.

seanmcdirmid 7y ago

I opted for an easy take home from Twitter recently when applying for a job there. However, the problem was so horribly written and vague that I couldn't figure out what they really wanted, even after asking questions. Honestly, I would have been better doing a problem with time constraints interactively, at least I could get some decent feedback about what was expected. Flubbing the Twitter take home worked out for the best, Google was a much better fit for me anyways. Their interview process was great also, and doing whiteboarding over VC affords certain advantages (getting to type in code, if only into Google docs, is a huge win for me, Jamboard + an iPad Pro worked out great as well).

anonymoushn 7y ago
When I interviewed at Google last year, they said they offered every candidate a Chromebook on which to type code instead of using a whiteboard, but the office I visited had no working interviewee Chromebooks. Maybe most people who go to on-sites can type code on a real computer.

baron816 7y ago

I think it would be interesting to ask a candidate what question they normally ask when they conduct interviews, and have them to walk through the problem. Has anyone here tried that or heard of that being done? I think you could potentially learn a lot more from that than you could with your own problem. First, there would be a lot less stress on them since it would be a problem they're presumably very familiar with. If they do a terrible job coding it up, well, that's a super clear signal. Second, stuff like how well the do setting up the problem, what kind of problem they choose (is it some generic Leetcode problem, or something they came up with themselves), what they look for in a good solution (just that it works/that it's over-optimized/obeys some arbitrary style standard/etc.) would all provide a less clear signal, but still useful information, especially regarding seniority.

LanceH 7y ago
That's such a loaded question, though. It is completely open ended. Will they be judged on the difficulty of the question? Their ability to answer it? Is it even emphasizing the right area for this interview? I strongly believe the candidate should be given every possible direction on what they are being judged on. I'll ask, "What are some of the things you look for in a code review?" If they hesitate or seem unsure about how to answer, I'll help them out with "What are some of the things that will get a PR rejected?" From there I try to open it up to a conversation rather than getting them to list the "correct" five things to cover in a code review.
pmiller2 7y ago
I couldn’t technically answer that question. My company uses a structured interview process, and all our questions fall under NDA.

srich36 7y ago

New graduate here who has this admittedly been on only one side of the interview process. In my interviews I got mostly white boarding problems and one take home. The author was mostly correct in discussing the advantages and disadvantages of both (namely the extra-time for take home assignments and archaic algorithms for white boarding) but I feel there are a couple points I can add. First, I actually believe white boarding questions (at least the automated kind) are assigned more indiscriminately than take home assignments. Many companies just send you online timed hackerrank or leetcode questions as a preliminary filter. Second, the entire interview process rewards people who study for the interviews rather than those who like to build projects. Unfortunately I believe companies miss out on a lot of qualified candidates, albeit reducing their false-positives from obscure algorithm-type problems. With so many applicants, reducing false-positives ensures quality hires so it works on the company’s end. Overall, my opinion is the interview process is relatively broken. Take-home assignments offer more insight into actual development skill, while whiteboard questions filter out a large portion of people (many who would be just as good, if not better, at actual implementations.) But it works for the interviewers so change seems unlikely. This is just my experience for new graduate interviews; perhaps it is different at higher levels.

ddrt 7y ago
It could reward those people because it exemplifies following a spec and completing to spec. The whole thing, studying and showing you will do things you don’t want to do (and do them well), and complete a really absurd set of requirements that have seemingly no value. However, you will stand out if you ask “why did you choose those questions?” Or “how did you come up with those problems?” Even the best interview that fits in all the check boxes trumps a similar candidate who shows they give a shit.
pdr2020 7y ago
Wouldn't this be fixed by presenting a on-site assignments? For example a small project, with test suites, where a feature or two needs to be completed and tests updated? (Disclosure: We did this at a company I worked at).
izacus 7y ago

jillesvangurp 7y ago

I've been asked to do coding interviews for freelance gigs a couple of times recently. This does not make sense to me. The whole point of freelancing is that either of us can terminate the contract on short notice if for whatever reason we don't like each other. Usually, if it comes up, I just recommend the company that if they have another qualified candidate (i.e. better qualified than me), they should do their best to hire them. It's a good way to cut the bullshit and get it in people's heads that they need to work hard to get me if they want me to commit to their project. If based on my profile they are not super eager to talk to me, I'm wasting my time because either they need somebody with a different profile or they are B's trying to hire C's. In both cases, that's a good reason for me to not engage any further. I've been in the interviewer role as well and I tend to not rely on coding exercises, whiteboard BS, etc. I usually just wing it knowing full well that interviews are in any case a bit of a crap shoot in terms of objective results. My goals when interviewing are very simple: 1) verify that what is in the person's CV holds up and try to get them to talk about what they did, how they did it, etc. 2) inject a few technical questions to see if they can come up with some good answers or whether they start bullshitting. 3) Figure out what they don't know and how they would deal with having to learn something new. 4) Figure out how well their experience aligns with my team's needs. I'm OK with a less than perfect match if a person has the potential to learn new stuff. Especially with junior roles, that's the most important thing because they probably don't know much yet. But really, I tend to go with my first impressions here and have learned the hard way that ignoring those leads to bad hires. Yes, there's a risk I'm wrong about not hiring somebody. But if it feels wrong, it's not going to happen. Usually, I know within a few minutes what it's going to be; the rest of the interview is then either about making sure the candidate is sold on the job or giving me enough material to defend the choice not to hire to my managers, HR, etc. Coding exercises don't really factor into this. If I have any doubts whatsoever about a person's ability to actually produce code, I'm not going to hire them. Unfortunately hiring is hard. There are a lot of mediocre candidates out there with a lot of mediocre recruiters representing them. I vastly prefer warm introductions through trusted friends and colleagues. So, as an interviewer, I deal with screening a lot of obviously not great candidates. The flip side is that when somebody good comes along, you need to be able to act fast because they'll have other options. Coding exercises are an obstacle for both sides.

atoav 7y ago

A technique that I learned from casting actors is: you don't have to be the candidates enemy to find out if they fit. If you ever find yourself trying to actively make the candidates life harder for no reason, you are doing it wrong (e.g. asking trick questions, expecting weirdly specific answers, etc) Instead give them multiple different hard (but realistic for the position) problems and be on their side every time to guide them. Try to get them to do the best they can and observe them all the time. This way you can get good feeling for how they are as a person, how they react to guidance and teamwork, and false negatives where a simple miscommunication leads to wrong solutions and a bad result, despite the candidate actually beeing good, are reduced. If they constantly and heavily rely on your guidance that is a bad sign. If they actively fight against your guidance, this is bad as well. But in the end you are ”casting” them for a certain position within an existing "cast", and sometimes a person who initially needs more guidance will do great after a while if they are in an environment where they can receive it, while people who actively avoid guidance could work just fine in a position where they don't get too much anyways etc. The opposite of this approach would be to ask an actor which pieces of [insert obscure european postramatic theatre director here] they know and if they know none assuming they can't act. Bonus points if the project doesn't have any connection to the topic asked at all. If anybody ever does this, it says more about the person asking, than the person answering. No — try really hard to get the best out of the candidates in the most accurate emulation of the job you want to give to them, and have a grasp of what they will get up to speed with in no time and what actually will get worse (e.g. personality traits). And if you areninsecure about someone give it another shot in a second round.

pmiller2 7y ago
I think you’ve got it right. The best interviews I’ve participated in felt like working with someone for an hour or so. As for the opposite scenario, you’ve got that right as well, except that the interviewer is a 25 year old, recently graduated theater major.
- atoav 7y ago
  > As for the opposite scenario, you’ve got that right as well, except that the interviewer is a 25 year old, recently graduated theater major. Insert there what you want, hehe. It always reminds me of one teacher I had, who always wanted to get oddly specific answers on his (always a little to unspecific) questions. He was so happy when nobody could answer it, he probably forgot what his job actually was. Behaviour like this tells more about the psyche of the interviewer than it tells about the candidate. Sometimes a behaviour like this is also the interviewers way of making sure the candidate doesn't expose them as the fraud they deep down feel they are: "See they didn't know that oddly specific bit (that I know!), so they must be worse than me – thank god!" In case of my teacher he felt insecure enough to feel the need to one-up teenagers and instead of teaching them something. Interviewers are only human after all as well. But it shouldn't be the candidates job to rub interviewers in the right way in order to make them comfortable with their own shortcomings as it shouldn't be the pupils job to massage teachers into actually teaching them something.

godot 7y ago

I really enjoyed this article. It's probably one of the most fair, unbiased, and insightful articles written on tech interviews and may be the best one I've read. Over the past 15 years, I have interviewed with many different companies. I once read that when you're a good fit for the company, the interview will feel suspiciously easy. This has been the case for all the times I've gotten offers. Reading this article, a lot of the good points remind me of a particular interview I had though. It was at relatively large tier-2 tech firm a few years ago (not the FAANG tier); one where I didn't get an offer, but the interview process and experience was memorably good. I remember specifically telling the recruiter even after the rejection that I really enjoyed the interview there. One of the rounds was with the principal engineer at the company, who was the most senior engineer at the company at the time if I recall correctly. It was the system architecture interview. I was asked to prepare by thinking of a system I had designed and architected in my past experience, and present it as if the principal engineer was my coworker and we would hold tech discussions. It's the closest thing I've ever gotten to the "Present code they wrote" idea that Andrew mentions here. For the coding round of this interview, I was asked to bring my own laptop (another point that Andrew mentioned here; this was also the only company of all the interviews I've had in 15 years that asked me to do this.). I was then given a problem at the beginning of that interview, and given 45 minutes or so to write code on my own (the interviewer actually walked out of the room and left me alone while I did). It was a reasonable problem to code for the allotted time, and had reasonable complexity but not overly difficult. When time's up, the interview came back and we discussed my code. Both of those rounds were something that was so well done that I think it was the most fair way a company could have assessed me correctly. I did well on the coding round, but my experience in system architecture was not a good match for the type of work they did, and in retrospect I could tell. Since I was interviewing at the Staff Engineer level, I think they correctly evaluated me and turned me down for the role. (Although I would point out that after that day of interview, they ended up being undecided, and asked me to come back one more time to have one more interview round, with one of the first engineers of the company there. I did poorly in that round for a variety of reasons, and that was ultimately what got me the rejection.)

youeseh 7y ago

Whiteboard interviews all the way. They are interactive. And they can be two-way interviews. All participants of the interview are interviewing each other. Take homes are basically free labor.

toolslive 7y ago

Why don't they trust the universities? If the candidate has a degree in computer science shouldn't he be able to code ?

OutThisLife 7y ago
Not in my experience.
- pkolaczk 7y ago
  The same applies to algorithmic challenges given for whiteboard coding. The fact one can reverse a linked list doesn't mean one would be a good developer in a real world project. Whiteboard coding doesn't check the basic system design skills.
  - dragonwriter 7y ago
    Whiteboard coding isn't designed to find good all-round developers, it's designed to filter the candidate pool to decrease the likelihood of candidates being selected that don't have the necessary coding chops. Other aspects of the selection process check other aspects of the desired skill set.
repiret 7y ago
While I think that ought to be a requirement for getting a CS degree, not all universities do. Moreover, being able to code isn't a boolean - while some people are indeed totally incompetent, among the competent people, some are better than others, and some have strengths and weaknesses that would make them more ore less suited to a particular role. It behooves an employer to try to measure these things.
kortilla 7y ago
Computer science doesn’t teach you how to write code.
rsynnott 7y ago
... No.

lifeisstillgood 7y ago

Is hiring broken for non-software jobs like chemists, physicists? What about writers or journalists? I only have vague insights into these industries but science jobs afaik rely a lot on recommendations - on calling prior collegues and asking. And for writers - well reading their previous stuff works well (easier with open source stuff) So i would suggest right approach is more a simple FizzBuzz screen, a cultural fit (careful of biases!) and then reading their past OSS / other work (so maybe a prepared portfolio is a good "take home") and calling their references.

ProZsolt 7y ago
then reading their past OSS / other work A lot of people don't have any polished open source code. I have a lot of stuff on Github, but I wrote it for myself. I don't care if it brakes If I only using it once or twice a year.

deleted 7y ago

[deleted]

topkai22 7y ago

As an interviewee, I have a particular distaste for take home questions, at least if given before doing a couple phone screens- I want to see a company invest time in me a bit before spending half a day coding. Besides, having been an interviewer more than an interviewee at this point I find being able to competently discuss previous projects with a degree of excitement more predictive of success anyway. I still ask to see interview coding, but it's more a check box for basic skills than a major focus

stared 7y ago

While the example was bad (too general, a potential for copy&paste from somewhere else), it was certainly not for the reason OP wrote: > The exercise was supposed to take "a few hours" according to the hiring manager, but I'm extremely rusty with ASP. Well, if they look for an ASP expert, this self-selection saves time on both sides! I use take-home assignments with boilerplate code, to reduce the initial time investment and make it easier to compare parts of the projects. (Otherwise, it may be a too open-ended task.) I try to make sure that the skills I require the skills I need. For example, when I needed a deep learning engineer (for a PyTorch project), I expect fluency in at least one deep learning framework. I made it explicit that instead PyTorch boilerplate then can use any other framework (a few people send in Keras). Yes, people can Google things (good!). And it some sense it balances more junior people (but willing to put more effort) with people with more experience. The later could be told easily by interview. When I ask questions about their solutions, it is easy to see which parts are done with understanding, which - not.

gwbas1c 7y ago
OP here. The reason why the stack requirement bothered me was that, during the phone screening, I basically told the hiring manager that take-homes are fine, but I walk away from anything that has a laundry list of required frameworks. They were looking for a very senior developer, not someone who would come in and know their stack on day 1. In my case, the problem wasn't ASP, it was all the other frameworks and patterns they wanted.

lmilcin 7y ago

From the point of view of interviewer. I find that take-home exercises have two major flaws: 1. The push the effort onto interviewee. Interviewee should not be punished for the interviewer's inability to prepare good questions and exercises and inability to design the process so that bad candidates are filtered out early in the process. 2. They are not really meaningful. I've been there as a candidate. It is difficult to voluntarily stick to the timelines given when you know you could do this or do that to improve the solution and that other candidates will not hesitate to do it. As an interviewer, I have no idea if the candidate did the exercise by themselves or with outside help. Some (not all) candidates will do anything to game the system. You don't want to get these people onboard. Based on that I am strongly against take home tasks. On the other hand I like whiteboard exercises because they let me discuss things in an environment and closely to circumstances we could be having at work. It is normal for me to draw algorithms or even implement pieces of (pseudo) code on the whiteboard and to use it for discussion. What I try to keep in mind that different people deal differently with the whiteboard. Some people like to visualize, some think a little bit more abstract. I had to work hard to be able to draw anything in a way that will be digestible by other people and I still think am doing only mediocre job of it.

mrfusion 7y ago

I’ve always wondered how mechanical or civil engineers are interviewed? How about accountants? You never hear about interview problems for them. As an industry we should take our dignity back.

benstrumental 7y ago
Those professions usually require a license gated by a standardized exam, which ensures some level of competency before showing up to the interview.

bitwize 7y ago

Back when I used to do whiteboard interviews, the trick was not to grade the candidate on right or wrong answers, but to look for signs of engineering thinking. The problem was simple, but well-suited to low-level languages like C, and we would even give them hints if they forgot some aspect of C-family syntax. BSers would reveal themselves in obvious ways, and anyone who displayed engineering competence was still in consideration.

tcrow 7y ago

We have a big app and need people who can come in and get up to speed very quickly. A solution to the interview problem that I've adopted over the last few years, to great success, is to conduct a 2 hour, 2 phase interview. - The first hour is spent having the candidates work through a problem in a sand boxed environment using the actual code base. - The second hour is spent recapping the work session, discussing their professional background, and general questions about other things they should know to be successful on our team. The coding test is designed to be solvable within the allotted time and without explicit knowledge of the business rules in the app and only requires an understanding of the actual technology they purport to have the required experience in. Running this test is a great way for us to understand what the candidate's skill level is and what problem solving approaches they use in a real-world scenario. They are free to talk about what they are doing (but is not required) and of course we offer guidance if they get stuck on something. Solving the test is NOT a pre-requisite for getting hired. In order to reduce anxiety for the candidate, I like to frame the test by telling them to imagine themselves as a short-order contractor who is coming in to help us with a problem that we are having a hard time solving ourselves. I find that this really helps. The feedback from the candidates for this testing style has always been positive, and I can tell you from post-hire experience, that it has helped us to bring some great developers on board. So far, there have been no false positives. In fact, it's so popular that other teams have either asked me to conduct their interviews or plan on adopting the same methodology for conducting theirs. If your struggling to get more consistency from your hiring approach, it might be worth a try.

geebee 7y ago

One thing I prefer about whiteboard coding interview exams is that the do require an investment of time from the interviewing company. Take-homes allow them to externalize the horrendous inefficiency of the interviewing process. Beyond that, I think articles like this are important to read and circulate widely. Young people need to know that if they become doctors, lawyers, nurses, actuaries, licensed processional engineers, dentists, or financial planners, they will indeed need to take a test. However, the test will be managed by a central authority, with predictable content, with a clear study path, and once you take the entrance exam, you won't have to take it over and over during job interviews for the rest of your life. There may be refresher exams, but again, they will be managed by the licensing body and will provide a clear preparation path, and will be graded consistently. It won't all be left to the whims of whoever thought up a question. If you become a dev, you will deal with interview exams, often conducted by people who are not really qualified to conduct exams, every time you interview, even if you have decades of experience. If you are a senior actuary, you will not need to integrate by parts every time you interview, but if you are a developer, you may very well have to find all matching subtrees in a binary tree every time you interview. You are signing up for a lifetime of retaking your intro to algorithms midterm. You may be able to avoid this by starting a consulting company, or going into management, but at this point, if you plan to be a "knowledge worker", I would highly recommend taking a look at other fields.

samsquire 7y ago

I feel that we (techies) make hiring miserable by asking for the perfect versions of ourselves. The result is entry level inflationism.

SergeAx 7y ago

When I'm giving candidate a take-home assignment - I'm always asking for time and effort estimation, both "net" and "gross", like in "how many hours will it take" and "when it will be ready". Actually, ability to give viable estimation and stick to it is more important than coding itself. As for other matters in article, my current company just paying candidate for finished assignment, plain and simple.