8 ms·

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as poss

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hours of your time it is only fair that I also spend this time with you. And also give you a chance to learn about me, your future boss. Not to mention I can learn more about you working with you for half an hour than I would ever learn by studying the code you might or might not have written yourself.

stephenhogsten 4y ago
These are great points. How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment? Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.
- kevmo314 4y ago
  How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.
  - smegsicle 4y ago
    and however it was enforced, it would increase the stress a great deal
    - stephenhogsten 4y ago
      True. More stress than an untimed take-home, but I would think less stress than a normal interview (for most engineers, at least).
      - ghaff 4y ago
        And the reality is that a lot of/most people interviewing in those type of situations arre very well familiar with time-boxed exams at school. (And even with take homes the reality of final exams as I recall is that you mostly didn't have the hours in the day to spend an outsized amount of time on one course.)
  - inanutshellus 4y ago
    Presumably the difference is that the interviewee gets to choose when to start the clock.
    - stephenhogsten 4y ago
      That's what I had in mind. You would need a tool to enforce it. At that point "take-home" may not be the best name anymore, but "remote interview" just sounds like it's an interview over video. Maybe "async interview"?
      - filoleg 4y ago
        It is already a thing. I had a 2hr take-home before that was utilizing hackerrank. But it wasn't a leetcode-style one (which are common), it was a skeleton of a React app, and I had to implement certain methods to tie it all up together. The timer would start ticking from the moment I click "begin", and it will warn you that the timer will start after you attempt clicking (so that you don't trigger the timer by accident).
  - ghaff 4y ago
    It doesn't need to be proctored. It would be easy to start a remote clock and the solution needs to be in by x minutes after the start. It can even be pretty generous. The idea is presumably not to see how fast they can sprint but just put some time-boxing in place so some people aren't taking days. People can get help of course but that's going to be the case absent effectively a remote proctored assignment. And now you're back to effectively in-person.
    - thaumasiotes 4y ago
      > People can get help of course but that's going to be the case absent effectively a remote proctored assignment. And now you're back to effectively in-person. I would vastly prefer an "in-person" interview in which I wasn't expected to speak to the interviewer to the in-person interviews we actually have. There is a world of difference.
      - ghaff 4y ago
        You don't want to ask questions?
        thaumasiotes 4y ago
        Why would I want that?
        ghaff 4y ago
        Because it's an opportunity to learn more about the company beyond what's on their website? Honestly, if I interviewed someone and they had no interest in asking any questions, that's probably an easy reject.
  - lenzm 4y ago
    I have used https://takehome.io/ https://takehome.io/. The timer starts when they check out the repo containing the project and instructions.
xtracto 4y ago
As a hiring manager I wholwheartedly agree with this. Take home tests introduce so many uncontrolled variables, I wouldn't be able to measure and compare two candidates.
zaphar 4y ago
Counter-Point. If the assignment is properly sized: no more than an hour. Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work. This is an interview after all. The power balance whether intended or not is very much different than an actual peer review If a candidate chooses to burn a whole week working on it that can be safely left up to them I think. But a take home test is not unfair because they decided to burn that much time on it. It is an attempt to let them give the clearest most accurate signal of their skill level and approach to problems. I would argue that not even pair programming accomplishes that as well as a take home assignment.
- NeverFade 4y ago
  > Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work. Or, they don't even do the task, but pay someone else to do it for them, so eventually all these take-home tests just go to the same set of expert test-takers who already solved all these tasks. Very efficient, and also worse than useless as a screening tool: the dishonest candidates most willing to cheat will score the highest. > If a candidate chooses to burn a whole week working on it that can be safely left up to them I think. But a take home test is not unfair because they decided to burn that much time on it. Of course it is. Worse: it creates adverse selection, from the interviewer perspective. Top candidates who are looking for their next job while working a demanding full time job will not be able to dedicate more a minimum of time to each of the many take-home tests they are asked to do. Meanwhile, desperate poor candidates will dedicate x10+ the time (or simply cheat) to pass. So as an interviewer, you'll end up passing the random desperate candidate who somehow got through your earlier screening, and any good candidate with multiple other options and commitments will submit work that looks far inferior.
  - glenjamin 4y ago
    If I set a take-home test designed to take two hours, and then very clearly tell the candidate to take two hours, and then they clearly spent way longer than that, then I will consider that a signal that they don’t follow instructions well and don’t value their time appropriately.
coxmichael 4y ago
How do you structure the 2 hour session? Would be interesting to hear which kinds of tasks you use in a short space of time.
- alfalfasprout 4y ago
  One of my favorites is a code review + take an existing codebase and add some functionality to it or fix some design issue with it.
blueberrychpstx 4y ago
Best of both worlds to use this compilation as inspiration for one’s own company pie programming sessions. I don’t find it useful to evaluate whether or not a potential hire can map and reduce a list in JavaScript or fix some silly react bug thags so obvious it hurts. Seeing how they approach real problems, architecting solutions on the fly etc are things I enjoy doing but I’m not as much on the hiring side so YMMV
1123581321 4y ago
A two hour pairing session is hard on people who need to fit their interviewing time into odd hours. It’s also stressful, because it’s so much more consequential than a session of real work with two peers. Not to say you shouldn’t do it—just keep in mind it also has trade-offs. And I soundly agree with the positives you list, especially that it forces the interviewer to commit time like the interviewee. I did a one hour take-home for my current job. I could see someone taking 2-3 hours on it if they were really rusty, and I think that is a good thing because rusty developers can be very good once back up to speed in a language or domain. Live programming wouldn’t give them a chance.
- twawaaay 4y ago
  If you can't find 2h for a Zoom call I start questioning your commitment to get the job, especially when there is no time limit on when you can have the meeting (within reason). When I had an important interview in a place where I really wanted to work I would usually take entire day off and make sure I am well rested, prepared and in right state of mind. Yes, it is stressful. I try to make interviews less stressful but I also believe ability to perform under stress is a desirable trait. I have been presented with many stressful situations in my past where I had to deal with high stakes, difficult problems on a short notice, with important people watching my every move. Like dealing with an outage that incapacitated an entire national bank and was causing a loss of about 50M USD per day. Now, I am mostly interviewing senior devs, team leads, tech leads, etc. I would be much more lenient with junior roles but I just don't have time to interview everybody and I also like to give the chance for other people to work on their interviewing skills.
sgarland 4y ago
The two things I like about them from a candidate perspective are that they're async, and that they generally have an actual problem to solve, rather than "solve N-Queens." Some companies (Dropbox comes to mind) have coding tests, but they're async, and they also aren't convoluted Leetcode problems. I've also interviewed people using both live and a take-home review, and don't have much of a preference there. If the former, I can poke your thoughts in realtime, give hints, etc. If the latter, I get to deep-dive into some of your design decisions.
- thaumasiotes 4y ago
  > they generally have an actual problem to solve, rather than "solve N-Queens." Some companies (Dropbox comes to mind) have coding tests, but they're async, and they also aren't convoluted Leetcode problems. I applied to Dropbox some years ago, and the coding test was an NP-hard problem: https://en.wikipedia.org/wiki/Rectangle_packing https://en.wikipedia.org/wiki/Rectangle_packing
  - sgarland 4y ago
    Oh my. Mine was SRE-specific to be fair, and was the kind of thing I'd do for fun to answer "what if..." It was timeboxed to two hours, and IIRC I had a working solution in about half that, then spent the rest iterating and improving.
ghaff 4y ago
There are a number of choices. 1.) You can treat it like pretty much any other engineering job and give a "normal" interview, which will vary a lot by company. You probably rely more on credentials than is sometimes the case with software development. 2.) You treat it more like writing, graphic design, video, photography, etc. and expect to see a portfolio. For fields like that, you might talk about thought processes or how they approach their work but the portfolio isn't really negotiable. 3.) In the case of software, you can maybe have someone effectively create a portfolio for the interview but now you're back to someone putting a lot of time into creating something--albeit possibly something reusable.
alfalfasprout 4y ago
100%. The signal you get from pair programming is enormous. You not only get to see the end product but how they think. Also it's a good gauge if the person is effective at communicating/working with others.
spmurrayzzz 4y ago
If the take home assessment is scoped to a small amount of time (1-2 hours is small IMO) with the expectation that the seniority level you're selecting for could complete it in less than that time, I don't see any problem as long as you don't put arbitrarily constrained time limits on how long you are willing wait for the submission. I agree with you that if the candidate is going to spend the time to do the exercise, the hiring manager should be spending an equal amount of time as well in the process. But that effort can come from the detailed review of the code, the time commitment of which should be in the service of being able to have a meaningful conversation about the candidate's submission in later rounds (which is useful for your last point about discovering what its like to actually work with the candidate).
LewisJEllis 4y ago
Have you ever considered that solutions to your perceived problem might exist? I’ve done multiple different takehomes with some kind of time limit in place to avoid that sort of thing. Tools have this built in. Also, there are definitely candidates who would prefer a takehome. If you’re worried that a takehome might be unfair, you can offer folks a choice to either do the exercise on their own async, or to do it live pairing with you. You might be surprised by how many will choose the former.
- alexyang21 4y ago
  I spoke to a head of engineering yesterday who asked candidates pick from 3 options: 1. Bring some work that you did previously (>4 hrs) and we'll discuss it. 2. Complete a 1-hour take-home. 3. Complete the 1-hour take-home live with one of our engineers. He said that 98% of candidates chose option #2. I was surprised at how high it was!
jakearmitage 4y ago
Do you think that spending 2 hours of someones time, under the scrutiny of a stranger and already under the stress of looking for a new job, is fair? Has anyone ever asked you to code for them, instead? So they could learn about you and how you deal with problems? Have they asked how you deal with situations under stress? Or maybe they never did so because they were never comfortable enough to do so, since they were in the spotlight?
aqme28 4y ago
Unless you pair at your job, I disagree with this. Emulate your actual work environment as much as possible. That’s what you’re actually trying to get signal about.
gchamonlive 4y ago
> I can learn more about you working with you for half an hour than I would ever learn by studying the code That's the problem. I think everyone is too focused on technical prowess and forgetting something just as important which is written documentation and explanation. Pair programming has the downside of selecting people with good interpersonal skills and will tell you nothing about quality of code delivered. A good middle ground I believe is simple takehome tasks with focus on documentation that could be done in no more than two to four afternoons.
- BlargMcLarg 4y ago
  >I think everyone is too focused on technical prowess >and will tell you nothing about quality of code delivered. These statements seem a little contradictory or I'm misunderstanding the context.
  - gchamonlive 4y ago
    Not being too focused on technical prowess doesn't mean being oblivious to code quality, so no I don't think they are contradictory.
    - BlargMcLarg 4y ago
      And you don't think that wording can easily be taken as "we focus too much on technical skills" while simultaneously saying "pair programming doesn't guarantee technical expertise"? Many people would think code quality is part technical prowess, at least. If anything it at least showcases the importance of clear communication and how easily things can get lost in the details.
      - gchamonlive 4y ago
        There is also the principle of the benefit of the doubt where, in doubt, assume the writer is not a complete idiot. Goes a long way to filling these communication details without forcing the writer to write a philosophical thesis.
anchochilis 4y ago
As a candidate, I vastly prefer a take-home to live whiteboarding or pairing. That background sensation of being observed and evaluated greatly interferes with my ability to concentrate and perform at my best. (I'll note that I have no problem pairing in real-world work scenarios! I love discussing problems with my team. It's the artificial pressure injected by the interview process that really gets to me)
- Ancient 4y ago
  Perfectly articulated. Thank you! +1 Vote
- 120bits 4y ago
  I have been in situations were I had to pair program for an interview. The artificial pressure is true, I don't know how to react and even speak out when I'm trying to show my coding skills. I don't have issues with whiteboarding and pair programming with my current teammates. I can speak up and have enough confident to do. But its not the same in an interview situation. I prefer take-home as well.
- alexyang21 4y ago
  How would a take-home compare to a hybrid format? For example, if you were given a 1-hour take-home followed by a live session asking about your thought process and then pairing to extend what you previously worked on.
  - anchochilis 4y ago
    I think a hybrid format is the right way to do take-homes. The candidate is less likely to resent the time invested in the take-home if they have a chance to discuss or "show off" their work. It also gives candidates who have less free time to complete a polished solution the chance to say "if I had more time, I would [add tests, optimize this function, ...]>" I still struggle with the "observer effect" during live extension exercises, but it's less pronounced than when starting with a problem from scratch. If I prepare a high quality solution in advance and do well in technical discussions, that's usually enough to offset any fumbling around during the live programming session due to interview jitters.
ArrayBoundCheck 4y ago
What about a take home test that should take 15min or less? Something not as simple as fizzbuzz but still takes < 15 lines of code?
alexfromapex 4y ago
Have been doing SE for 10 years now. Got burned on three of these recently because they didn't used to be as common. Talking several hours each one. It leaves you with the feeling that the company is stealing your work. Will not repeat. Also, one company had me do a SQL take home which I thought was amusing because then one of the employees nit-picked all the SQL code which I hastily wrote because it was already a risky time investment, even though it was a Python job and I'd likely just use an ORM. I would argue that a truly qualified interviewer should be able to have a dialogue where they ask you very pointed questions and give you maybe a 20 minute coding problem. If they're not satisfied, they should invite you back for another interview. The major drawback of take-home tests is they are usually very subjective and huge time investments. I ultimately got 3 job offers, got to the final round at 3 others which had take homes, and accepted one at a FAANG company who didn't use these poor interview tactics but to each their own.
skhameneh 4y ago
After doing a few take home assignments, I've learned many won't give the courtesy of Q&A afterwards; many won't give any feedback by default. I loved giving live code sessions, the more open-ended the better. Give an objective, explain there's no wrong answers, answer all questions, give answers when asked, keep it simple, and gauge response. I found it exposed attitude and knowledge.
- alexyang21 4y ago
  I blame the legal teams at large companies who are worried that a well-meaning hiring manager will give feedback that could be used in a lawsuit. Love that your approach focuses on understanding thought process! IMO many companies focus too much on raw technical skills, when softer skills like attitude may be more predictive of on-the-job performance. My team is working to elicit the same signal in a take-home format (since it's more scalable), but I think the best is a combination of the two: short (~1 hr) take-home + follow-up live session on the work that was already started.
lapser 4y ago
If you give every person on the street 2 hours of your time, you wouldn't be able to do your day job. I don't like take home tests either, but the point of them is removing the bad candidates from the pipeline so that people who interview them (often ICs, and these are some of the biggest expenses within the company) don't waste time on someone that is unlikely to be qualified. Personally when I'm interviewing and I get take home tests I choose my battles. I only do them if I really want the job. If it is worth my time. It's unfortunate, but take home tests are a necessary evil until something better comes along.
- yeeyeeyee 4y ago
  If only there were this magical piece of paper that everyone presented when they applied listing all their relevant skills and experience so that the company can determine if it's worth giving them 2 hours of their time. All kidding aside, the efficient thing about resumes is that you write them once and use them everywhere. Coding samples are inefficient because you have to take 4h to write one for each company. Edit: the other problem with takehomes is that it self-selects the people who have scarce-enough opportunities that they find it worth their time to do them.
  - lapser 4y ago
    Right, and the issue with that is that plenty of people lie, and even more exxagerate what they did.
PUSH_AX 4y ago
A two hour coding session is a nope from me, and a lot of talent will feel the same, you're going to miss out on some good people. You're going to learn little to nothing regardless of whether you're face to face or not. Modern interview processes are an exercise in feeding the other side things they want to see/hear. I'm a firm believer that good fit cannot be determined until the candidate has been in the seat for a month or so.
MehdiHK 4y ago
One solution that seemed to be the best of both is, a take home test followed by pair programming session to add a small feature on top of that, or refactor something, or write tests etc. Real world challenges needs time to think, often time to refactor after initial working prototype. A take home allows that. Also the candidate is already relaxed in the pair programming session because it's a familiar codebase. You'll get signal if they didn't write it or wrote something they didn't quite understand.

stephenhogsten 4y ago

These are great points. How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment? Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.

kevmo314 4y ago
How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.
- smegsicle 4y ago
  and however it was enforced, it would increase the stress a great deal
  - stephenhogsten 4y ago
    True. More stress than an untimed take-home, but I would think less stress than a normal interview (for most engineers, at least).
    - ghaff 4y ago
      And the reality is that a lot of/most people interviewing in those type of situations arre very well familiar with time-boxed exams at school. (And even with take homes the reality of final exams as I recall is that you mostly didn't have the hours in the day to spend an outsized amount of time on one course.)
- inanutshellus 4y ago
  Presumably the difference is that the interviewee gets to choose when to start the clock.
  - stephenhogsten 4y ago
    That's what I had in mind. You would need a tool to enforce it. At that point "take-home" may not be the best name anymore, but "remote interview" just sounds like it's an interview over video. Maybe "async interview"?
    - filoleg 4y ago
      It is already a thing. I had a 2hr take-home before that was utilizing hackerrank. But it wasn't a leetcode-style one (which are common), it was a skeleton of a React app, and I had to implement certain methods to tie it all up together. The timer would start ticking from the moment I click "begin", and it will warn you that the timer will start after you attempt clicking (so that you don't trigger the timer by accident).
- ghaff 4y ago
  It doesn't need to be proctored. It would be easy to start a remote clock and the solution needs to be in by x minutes after the start. It can even be pretty generous. The idea is presumably not to see how fast they can sprint but just put some time-boxing in place so some people aren't taking days. People can get help of course but that's going to be the case absent effectively a remote proctored assignment. And now you're back to effectively in-person.
  - thaumasiotes 4y ago
    > People can get help of course but that's going to be the case absent effectively a remote proctored assignment. And now you're back to effectively in-person. I would vastly prefer an "in-person" interview in which I wasn't expected to speak to the interviewer to the in-person interviews we actually have. There is a world of difference.
    - ghaff 4y ago
      You don't want to ask questions?
      - thaumasiotes 4y ago
        Why would I want that?
        ghaff 4y ago
        Because it's an opportunity to learn more about the company beyond what's on their website? Honestly, if I interviewed someone and they had no interest in asking any questions, that's probably an easy reject.
- lenzm 4y ago
  I have used https://takehome.io/ https://takehome.io/. The timer starts when they check out the repo containing the project and instructions.

xtracto 4y ago

As a hiring manager I wholwheartedly agree with this. Take home tests introduce so many uncontrolled variables, I wouldn't be able to measure and compare two candidates.

zaphar 4y ago

Counter-Point. If the assignment is properly sized: no more than an hour. Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work. This is an interview after all. The power balance whether intended or not is very much different than an actual peer review If a candidate chooses to burn a whole week working on it that can be safely left up to them I think. But a take home test is not unfair because they decided to burn that much time on it. It is an attempt to let them give the clearest most accurate signal of their skill level and approach to problems. I would argue that not even pair programming accomplishes that as well as a take home assignment.

NeverFade 4y ago
> Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work. Or, they don't even do the task, but pay someone else to do it for them, so eventually all these take-home tests just go to the same set of expert test-takers who already solved all these tasks. Very efficient, and also worse than useless as a screening tool: the dishonest candidates most willing to cheat will score the highest. > If a candidate chooses to burn a whole week working on it that can be safely left up to them I think. But a take home test is not unfair because they decided to burn that much time on it. Of course it is. Worse: it creates adverse selection, from the interviewer perspective. Top candidates who are looking for their next job while working a demanding full time job will not be able to dedicate more a minimum of time to each of the many take-home tests they are asked to do. Meanwhile, desperate poor candidates will dedicate x10+ the time (or simply cheat) to pass. So as an interviewer, you'll end up passing the random desperate candidate who somehow got through your earlier screening, and any good candidate with multiple other options and commitments will submit work that looks far inferior.
- glenjamin 4y ago
  If I set a take-home test designed to take two hours, and then very clearly tell the candidate to take two hours, and then they clearly spent way longer than that, then I will consider that a signal that they don’t follow instructions well and don’t value their time appropriately.

coxmichael 4y ago

How do you structure the 2 hour session? Would be interesting to hear which kinds of tasks you use in a short space of time.

alfalfasprout 4y ago
One of my favorites is a code review + take an existing codebase and add some functionality to it or fix some design issue with it.

blueberrychpstx 4y ago

Best of both worlds to use this compilation as inspiration for one’s own company pie programming sessions. I don’t find it useful to evaluate whether or not a potential hire can map and reduce a list in JavaScript or fix some silly react bug thags so obvious it hurts. Seeing how they approach real problems, architecting solutions on the fly etc are things I enjoy doing but I’m not as much on the hiring side so YMMV

1123581321 4y ago

A two hour pairing session is hard on people who need to fit their interviewing time into odd hours. It’s also stressful, because it’s so much more consequential than a session of real work with two peers. Not to say you shouldn’t do it—just keep in mind it also has trade-offs. And I soundly agree with the positives you list, especially that it forces the interviewer to commit time like the interviewee. I did a one hour take-home for my current job. I could see someone taking 2-3 hours on it if they were really rusty, and I think that is a good thing because rusty developers can be very good once back up to speed in a language or domain. Live programming wouldn’t give them a chance.

twawaaay 4y ago
If you can't find 2h for a Zoom call I start questioning your commitment to get the job, especially when there is no time limit on when you can have the meeting (within reason). When I had an important interview in a place where I really wanted to work I would usually take entire day off and make sure I am well rested, prepared and in right state of mind. Yes, it is stressful. I try to make interviews less stressful but I also believe ability to perform under stress is a desirable trait. I have been presented with many stressful situations in my past where I had to deal with high stakes, difficult problems on a short notice, with important people watching my every move. Like dealing with an outage that incapacitated an entire national bank and was causing a loss of about 50M USD per day. Now, I am mostly interviewing senior devs, team leads, tech leads, etc. I would be much more lenient with junior roles but I just don't have time to interview everybody and I also like to give the chance for other people to work on their interviewing skills.

sgarland 4y ago

The two things I like about them from a candidate perspective are that they're async, and that they generally have an actual problem to solve, rather than "solve N-Queens." Some companies (Dropbox comes to mind) have coding tests, but they're async, and they also aren't convoluted Leetcode problems. I've also interviewed people using both live and a take-home review, and don't have much of a preference there. If the former, I can poke your thoughts in realtime, give hints, etc. If the latter, I get to deep-dive into some of your design decisions.

thaumasiotes 4y ago
> they generally have an actual problem to solve, rather than "solve N-Queens." Some companies (Dropbox comes to mind) have coding tests, but they're async, and they also aren't convoluted Leetcode problems. I applied to Dropbox some years ago, and the coding test was an NP-hard problem: https://en.wikipedia.org/wiki/Rectangle_packing https://en.wikipedia.org/wiki/Rectangle_packing
- sgarland 4y ago
  Oh my. Mine was SRE-specific to be fair, and was the kind of thing I'd do for fun to answer "what if..." It was timeboxed to two hours, and IIRC I had a working solution in about half that, then spent the rest iterating and improving.

ghaff 4y ago

There are a number of choices. 1.) You can treat it like pretty much any other engineering job and give a "normal" interview, which will vary a lot by company. You probably rely more on credentials than is sometimes the case with software development. 2.) You treat it more like writing, graphic design, video, photography, etc. and expect to see a portfolio. For fields like that, you might talk about thought processes or how they approach their work but the portfolio isn't really negotiable. 3.) In the case of software, you can maybe have someone effectively create a portfolio for the interview but now you're back to someone putting a lot of time into creating something--albeit possibly something reusable.

alfalfasprout 4y ago

100%. The signal you get from pair programming is enormous. You not only get to see the end product but how they think. Also it's a good gauge if the person is effective at communicating/working with others.

spmurrayzzz 4y ago

If the take home assessment is scoped to a small amount of time (1-2 hours is small IMO) with the expectation that the seniority level you're selecting for could complete it in less than that time, I don't see any problem as long as you don't put arbitrarily constrained time limits on how long you are willing wait for the submission. I agree with you that if the candidate is going to spend the time to do the exercise, the hiring manager should be spending an equal amount of time as well in the process. But that effort can come from the detailed review of the code, the time commitment of which should be in the service of being able to have a meaningful conversation about the candidate's submission in later rounds (which is useful for your last point about discovering what its like to actually work with the candidate).

LewisJEllis 4y ago

Have you ever considered that solutions to your perceived problem might exist? I’ve done multiple different takehomes with some kind of time limit in place to avoid that sort of thing. Tools have this built in. Also, there are definitely candidates who would prefer a takehome. If you’re worried that a takehome might be unfair, you can offer folks a choice to either do the exercise on their own async, or to do it live pairing with you. You might be surprised by how many will choose the former.

alexyang21 4y ago
I spoke to a head of engineering yesterday who asked candidates pick from 3 options: 1. Bring some work that you did previously (>4 hrs) and we'll discuss it. 2. Complete a 1-hour take-home. 3. Complete the 1-hour take-home live with one of our engineers. He said that 98% of candidates chose option #2. I was surprised at how high it was!

jakearmitage 4y ago

Do you think that spending 2 hours of someones time, under the scrutiny of a stranger and already under the stress of looking for a new job, is fair? Has anyone ever asked you to code for them, instead? So they could learn about you and how you deal with problems? Have they asked how you deal with situations under stress? Or maybe they never did so because they were never comfortable enough to do so, since they were in the spotlight?

aqme28 4y ago

Unless you pair at your job, I disagree with this. Emulate your actual work environment as much as possible. That’s what you’re actually trying to get signal about.

gchamonlive 4y ago

> I can learn more about you working with you for half an hour than I would ever learn by studying the code That's the problem. I think everyone is too focused on technical prowess and forgetting something just as important which is written documentation and explanation. Pair programming has the downside of selecting people with good interpersonal skills and will tell you nothing about quality of code delivered. A good middle ground I believe is simple takehome tasks with focus on documentation that could be done in no more than two to four afternoons.

BlargMcLarg 4y ago
>I think everyone is too focused on technical prowess >and will tell you nothing about quality of code delivered. These statements seem a little contradictory or I'm misunderstanding the context.
- gchamonlive 4y ago
  Not being too focused on technical prowess doesn't mean being oblivious to code quality, so no I don't think they are contradictory.
  - BlargMcLarg 4y ago
    And you don't think that wording can easily be taken as "we focus too much on technical skills" while simultaneously saying "pair programming doesn't guarantee technical expertise"? Many people would think code quality is part technical prowess, at least. If anything it at least showcases the importance of clear communication and how easily things can get lost in the details.
    - gchamonlive 4y ago
      There is also the principle of the benefit of the doubt where, in doubt, assume the writer is not a complete idiot. Goes a long way to filling these communication details without forcing the writer to write a philosophical thesis.

anchochilis 4y ago

As a candidate, I vastly prefer a take-home to live whiteboarding or pairing. That background sensation of being observed and evaluated greatly interferes with my ability to concentrate and perform at my best. (I'll note that I have no problem pairing in real-world work scenarios! I love discussing problems with my team. It's the artificial pressure injected by the interview process that really gets to me)

Ancient 4y ago
Perfectly articulated. Thank you! +1 Vote
120bits 4y ago
I have been in situations were I had to pair program for an interview. The artificial pressure is true, I don't know how to react and even speak out when I'm trying to show my coding skills. I don't have issues with whiteboarding and pair programming with my current teammates. I can speak up and have enough confident to do. But its not the same in an interview situation. I prefer take-home as well.
alexyang21 4y ago
How would a take-home compare to a hybrid format? For example, if you were given a 1-hour take-home followed by a live session asking about your thought process and then pairing to extend what you previously worked on.
- anchochilis 4y ago
  I think a hybrid format is the right way to do take-homes. The candidate is less likely to resent the time invested in the take-home if they have a chance to discuss or "show off" their work. It also gives candidates who have less free time to complete a polished solution the chance to say "if I had more time, I would [add tests, optimize this function, ...]>" I still struggle with the "observer effect" during live extension exercises, but it's less pronounced than when starting with a problem from scratch. If I prepare a high quality solution in advance and do well in technical discussions, that's usually enough to offset any fumbling around during the live programming session due to interview jitters.

ArrayBoundCheck 4y ago

What about a take home test that should take 15min or less? Something not as simple as fizzbuzz but still takes < 15 lines of code?

alexfromapex 4y ago

Have been doing SE for 10 years now. Got burned on three of these recently because they didn't used to be as common. Talking several hours each one. It leaves you with the feeling that the company is stealing your work. Will not repeat. Also, one company had me do a SQL take home which I thought was amusing because then one of the employees nit-picked all the SQL code which I hastily wrote because it was already a risky time investment, even though it was a Python job and I'd likely just use an ORM. I would argue that a truly qualified interviewer should be able to have a dialogue where they ask you very pointed questions and give you maybe a 20 minute coding problem. If they're not satisfied, they should invite you back for another interview. The major drawback of take-home tests is they are usually very subjective and huge time investments. I ultimately got 3 job offers, got to the final round at 3 others which had take homes, and accepted one at a FAANG company who didn't use these poor interview tactics but to each their own.

skhameneh 4y ago

After doing a few take home assignments, I've learned many won't give the courtesy of Q&A afterwards; many won't give any feedback by default. I loved giving live code sessions, the more open-ended the better. Give an objective, explain there's no wrong answers, answer all questions, give answers when asked, keep it simple, and gauge response. I found it exposed attitude and knowledge.

alexyang21 4y ago
I blame the legal teams at large companies who are worried that a well-meaning hiring manager will give feedback that could be used in a lawsuit. Love that your approach focuses on understanding thought process! IMO many companies focus too much on raw technical skills, when softer skills like attitude may be more predictive of on-the-job performance. My team is working to elicit the same signal in a take-home format (since it's more scalable), but I think the best is a combination of the two: short (~1 hr) take-home + follow-up live session on the work that was already started.

lapser 4y ago

If you give every person on the street 2 hours of your time, you wouldn't be able to do your day job. I don't like take home tests either, but the point of them is removing the bad candidates from the pipeline so that people who interview them (often ICs, and these are some of the biggest expenses within the company) don't waste time on someone that is unlikely to be qualified. Personally when I'm interviewing and I get take home tests I choose my battles. I only do them if I really want the job. If it is worth my time. It's unfortunate, but take home tests are a necessary evil until something better comes along.

yeeyeeyee 4y ago
If only there were this magical piece of paper that everyone presented when they applied listing all their relevant skills and experience so that the company can determine if it's worth giving them 2 hours of their time. All kidding aside, the efficient thing about resumes is that you write them once and use them everywhere. Coding samples are inefficient because you have to take 4h to write one for each company. Edit: the other problem with takehomes is that it self-selects the people who have scarce-enough opportunities that they find it worth their time to do them.
- lapser 4y ago
  Right, and the issue with that is that plenty of people lie, and even more exxagerate what they did.

PUSH_AX 4y ago

A two hour coding session is a nope from me, and a lot of talent will feel the same, you're going to miss out on some good people. You're going to learn little to nothing regardless of whether you're face to face or not. Modern interview processes are an exercise in feeding the other side things they want to see/hear. I'm a firm believer that good fit cannot be determined until the candidate has been in the seat for a month or so.

MehdiHK 4y ago

One solution that seemed to be the best of both is, a take home test followed by pair programming session to add a small feature on top of that, or refactor something, or write tests etc. Real world challenges needs time to think, often time to refactor after initial working prototype. A take home allows that. Also the candidate is already relaxed in the pair programming session because it's a familiar codebase. You'll get signal if they didn't write it or wrote something they didn't quite understand.