12 ms·

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of g

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a full-time job and a full-time life outside of work. Spending that much time to qualify for a single job is too much to ask of anyone. If it were to pass a generic proficiency certification applicable to many positions, I would consider it, but this does not scale if a candidate is applying for multiple positions.

odonnellryan 11y ago
I think it depends on the project. I'm looking to switch roles, and some projects are very small: solve a simple algo question. Should take less than an hour, which is less than the average interview, anyway! So no problem. Other projects are enormous. 40-80 HRs. of work, easily (write an entire web-app!) I did one of these... did not get the offer. I would not want to complete another.
eitally 11y ago
My wife is in this position now, and it's not for a programming job. She works in pharmaceuticals, in pharmacovigilance & risk management. She does a lot of data analysis & writing (the latest report she wrote for Health Canada & the FDA was 461 pages). A company she's interviewing with asked her to do a sample writing task as part of her application/interview. She hasn't decided whether it's worth it or not. For someone who has 15 years experience in the field, excellent recommendations and credentials, spending (her estimate) 12-15 hours doing pro bono work just to apply somewhere is ridiculous.
- tlrobinson 11y ago
  12+ hours is excessive even if you don't have 15 years experience.
- akg_67 11y ago
  I work in data analysis and I decline any "test" requests for a job or project. If you don't value my time before hiring, you definitely are not going to value my time after hiring. Also, if you are incapable of judging a fit based on resume, interviews, and just talking with me, you are not mature enough, experienced enough, smart enough and intelligent enough and have failed the test for being my potential coworker and boss. Edit: The tests make perfect sense for entry level positions and entry level candidates but once someone requesting tests for experienced positions and candidates, it shows the immaturity of the company, group, team and people who are unable to assess suitability of candidate and place too much emphasis on technical skills and tests as a crutch. The best jobs in my career were those where coworkers and boss had good relationship dynamics. Technical excellence had nothing to do with how well we worked and delivered results.
  - Buetol 11y ago
    A simple solution is for the company to offer you something for the time you spent doing the sample test (money, gift card, their product,...)
    - brudgers 11y ago
      How about a car? Then it is obviously worth the applicant's time and since a professional can deliver or destroy a Bimmer's worth of corporate value on any given decision it easily pays for itself in identifying good candidates and weeding out bad ones.
      - vorg 11y ago
        > it easily pays for itself in identifying good candidates and weeding out bad ones It even easily pays for itself in just weeding out the bad ones.
- technofiend 11y ago
  I had a similar experience last year applying for a major technology company. Although I still lightly code in the devops sense (process automation) and to do typical management data analysis my day job is managing people, processes and crises. If you have billion dollar trading systems and need a cool-headed guy who can talk to VPs or directors in one breath and SAs, DBAs, SAN, Network, etc guys in the next I'm your man. I applied to head a global support organization, with teams in (from memory) North America, India and Asia Pacific. I'm pretty sure day-to-day would be a mix of employee career management, forward planning with my application development partners to make sure we were prepared capacity and training-wise to take on whatever's next, and crisis management of production issues and the like. Business as usual as they say. Yet I found myself answering computational complexity questions about hash map insertions. To be fair, before I became quite so management-centric I've found plenty of badly performing code via the usual tools (dtrace, strace, thread dumps), and sql queries using analogous database tools, but at some point you have to rely on technical people to do their jobs. Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? Or am I clueless?
  - saidajigumi 11y ago
    > Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? An organization that does this is hiring stupidly. They have clearly demonstrated via interview that they don't understand the job to be done and/or how to determine that a candidate can do that job. Asking low-level technical questions is a terrible way to answer what I'll guess was the intended "question", something like: Does this candidate understand the technology and work well enough to make executive decisions grounded in our organization's reality?
  - Eridrus 11y ago
    > at some point you have to rely on technical people to do their jobs. On one hand, I totally agree that technical tasks should be left to the technical people, preferably those who are actually working on the system. There's almost nothing worse than a manager trying to take technical decisions out of people's hands. > Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? Or am I clueless? On the other, I think people want to work for people they respect and to the extent that people respect technical chops that might be what they are screening for. Though the hash map question sounds like a soft ball, so it might be screening for a baseline of software competence.
- IndianAstronaut 11y ago
  >spending (her estimate) 12-15 hours doing pro bono work just to apply somewhere is ridiculous. Keep in mind that in today's job market, there are probably 10 people lined up who are willing to do just that.
  - shredprez 11y ago
    Ah, but the willing aren't able and the able aren't willing. In any case, I'd avoid companies that recklessly fleece applicants just because they can. Once you've proven your worth professionally, begging for scraps is pathetic in any job market. For every greedy business with ten desperate applicants, there's an honest organization or untapped market ready to pay fairly for what value you offer. The question I consider when asked to perform for free: why should anyone simply donate their knowledge to a profitable business after spending a lifetime to attain it?
  - ankitml 11y ago
    Doing something because you can is the worst reason to do so.
- rwallace 11y ago
  Does she have existing samples of writing that she can point to, not locked up under copyright? If so, then I would point to those samples and suggest those should be relevant data for assessment and I'd like to get further in the process before putting in extra work for this. If they're prepared to discuss the matter, great. If they just give a blank 'sorry, this is our policy,' then walk away. Why? Because if they're not willing to have a discussion with you, that means they don't care about you. It means they think they have a hefty surplus of candidates. Which means you are very unlikely to be the chosen one, and you would probably just waste a lot of your time.
ammon 11y ago
Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview. I think this might help a lot of people (clearly we have to test this)
- rezistik 11y ago
  I personally prefer it to a point. My only request when I get this type of interview is that I'm allowed to make the project and requirements public on my github for future interviewers. If I'm doing a free project I feel it's fair.
  - ammon 11y ago
    Well, that's hard. Standardizing evaluation (critical to reducing bias) requires that everyone (or a lot of people) see a similar problem. And when someone is working on their own time, I think it's important that there's not working code easily accessible online. Perhaps this can be overcome by asking candidates to talk through their code after they are done
    - hayksaakian 11y ago
      if you change the assessment every so often, you could permit people to release their work eventually.
      - ammon 11y ago
        I like that idea!
      - Silhouette 11y ago
        If someone solved a significant problem at your request, but despite this you did not hire them or otherwise compensate them for that work, I don't see how you have any right either legally or ethically to expect them not to then use the work to advertise what they can do to anyone else. The premise is one-sided enough already (though the argument elsewhere in this HN discussion that it might be preferred by nervous interviewees is an interesting one I haven't noticed before) and I think expecting any sort of loyalty or confidentiality after the process has ended is rather optimistic.
Tech1 11y ago
I'm kind of in a similar boat. I've done them for companies that I like in the past, however I flat out refuse to do them for recruiting / job placement companies. ammon / uptown, I've put off doing my screen-sharing section of your interview process (I got pushed past the phone screen?) due to a lack of time. Maybe adding a way to 'schedule' it would be helpful for those of us that live by our g.calendars....and you all as well.
- Harj 11y ago
  What would be the best experience for you to schedule the interview? We'd definitely like to make that easier.
  - Tech1 11y ago
    Just something simple, allow me to add it to my google calendar.
geebee 11y ago
I did this once. After a take home quiz that took about an hour, I had a 6-7 hour "homework" problem. I did it, and sent it in. It took the company a month to get back to me, though I did stay in contact with the recruiter. The final answer, delivered by the recruiter, was "we've decided not to more forward at this time." No other feedback. I have no idea if anyone even looked at it - I never did speak to a dev at the company. This experience left me embarrassed, honestly. I don't think it reflects well on me that I put up with it even once. At this point, I will participate in tech interviews, but I won't do any more take-home assignments. I suppose this could change under different circumstances. Gayle Laakmann McDowell, who wrote "Cracking the Coding Interview", wrote an insightful post her blog about this. It's titled "Companies who give candidates homework assignments: knock it off!" http://www.gayle.com/blog/2013/09/18/companies-who-give-candidates-homework-assignments-knock-it-off http://www.gayle.com/blog/2013/09/18/companies-who-give-cand... She mentions something I see as a problem as well - these tests allow the company to burn a lot of a candidate's time, but not the other way around. I may have burned 6+ hours interviewing unsuccessfully at google, but they parted with 6+ hours of developer time as well (though I did spend a lot of additional hours reviewing algorithms and doing sample problems). This creates a natural set of checks and balances that don't exist with homework assignments. She does have some good suggestions about how to use a take home if a company is determined to do it. To me, a particularly insightful one was to have a high (she says 90%+) passage rate. I agree. If you're going to ask a candidate to spend a day on a homework assignment, you should be very close to an offer. If you're using it as an early filter, you are wasting a tremendous amount of developer time.
- jackieDQ 11y ago
  Yep I've been burned a couple times by this and only under certain circumstances will I do a take home project. Even worse is when they also ask before hand not to post it to your github account. If they aren't going to give me feedback, then I'm going to host it and seek it out elsewhere.
  - solve 11y ago
    Love that. If they burn too many people, they have to create new questions.
techiferous 11y ago
I've had a different experience on the hiring side of the table. The coding exercise that we provide as part of the interview has been very successful in surfacing the skill and talent of the programmer. By using that screening device, we have assembled a team of very sharp programmers. It's totally worth filtering out people who choose not to spend the time. Even though we end up losing some potential candidates, it's worth it for a 0% chance of a mishire for technical reasons.
- solve 11y ago
  A take-home test has a 0% chance of being cheated on? Are you sure you're talking about the same thing here?
  - techiferous 11y ago
    Our exercise can't be cheated on because we discuss it with the candidate during the in-person interview and even ask them to change the code in a simple way. Also, the exercise is designed to be original and so a solution can't be Googled.
    - solve 11y ago
      We're talking only about the "in their own time" take-home types of tests. The type of cheating I'm talking about is outsourcing the problem or even just having a friend do it for them.
- jrs235 11y ago
  We're currently hiring and for the first time, we should have started doing this years ago, given a simple coding exercise. I estimate it would/should take any programmer/problem solver no more than 15, maybe 20, minutes to come up with the solution if they have experience in the language we request they solve it in. The exercise doesn't require specific domain knowledge but should quickly assess whether a programmer, particularly with experience in a different language, is able to learn and apply the syntax and semantics of another language. If they are completely unfamiliar with the language then maybe an hour. They can use whatever references they want. We will check for complete copy/pasted code and alter the requirements and/or test data over time. The other great benefit is we get to see what/if/how they ask questions to clarify the problem and solution. EDIT: This exercise is geared towards fresh graduates or developers that don't have experience in the language we use. Can it be gamed? Yes. Might some people not bother? Yes. Are we okay with that? Yes.
solve 11y ago
My rule for these - only do these tests in-person, on-site, and after after the company has invested a significant amount of their employees' time to interview you in-person. Tons of companies hand out these "homework" tests to hundreds of people without having even the slightest intention of hiring them. F* that.
s0uthPaw88 11y ago
I feel that if you are asking a company to invest $100k/yr on you, it is not unreasonable for them to ask for a few hours of your time.
- ruswick 11y ago
  The point is that they pay you the $100k for the time you give them as an employee. They don't pay you a cent for the interview project, so you shouldn't give them a single second in return.
  - rqebmm 11y ago
    They don't pay you for the interview, but that usually costs the company several man-hours worth of work just for the time an applicant is on-site. Should applicants be reimbursing companies for failed interviews?
    - ruswick 11y ago
      Maybe. I've always thought it would be interesting to make applicants pay an application fee. This would cut down on people "spraying and praying" with their application, and would lessen the workload for companies. It would also justify spending more time and effort in reading applications, since the company isn't just wasting resources reading bad applications. Even under the current system, at least the waste is symmetric. I give up a few hours of my time, and the company gives up a few hours of its time. There's equity. The mutual work that the company and applicant do offset each other. A take-home test model skews that balance in favor of the company. An applicant can spend 10 or more hours working the project. The company can run it through an automated testing suite and have a recruiter spend five minutes looking it over. The system is designed to waste more of the applicant's time and less of the company's.
      - rqebmm 11y ago
        > Even under the current system, at least the waste is symmetric. But it's never symmetric. Someone in HR spent time setting up the job posting and managing the process. Someone in management took the time to respond to HR and review your resum, then approve the interview. At least one person at a time is in the interview with you. Then the entire team will spend time afterward breaking it down. A single hour spent by a bad candidate wastes at least 3 man-hours of work by the company, and most likely more.
ruswick 11y ago
Also keep in mind that applying to jobs is frequently throwing your application into a black hole and just praying. Sending a resume into the black hole is one thing. Sending a large codebase or project you worked on as part of the application is entirely different. I remember a few months ago, I was applying for an internship with a company and they requested that I write a web app to test coding proficiency before they thought about granting an interview. I spent about 12-15 hours writing the thing, deployed the app to Heroku and put the code on Github. I sent a link to the app and the repo to the recruiter. I never got a response of any kind. Although some companies do try to make the application experience as pleasant as possible for the applicant, the majority of companies don't give a shit about the people who want to work for them, and make no attempt to respect the time of their applicants. Placing greater demands on the applicant is an easy way to shift more of the workload from the recruiter to the applicant. The applicant is the loser in this situation.
dfan 11y ago
My last company had a screening coding problem that was designed to take around a day (although it didn't have to; when I did it as a reference point, it took a couple of hours). I was always kind of astonished how many applicants were willing to do the whole thing, given, as you say, that they often already had a full-time job and/or were applying for multiple positions. We always gave really extensive feedback on it, partially because one of the things were were looking for was how people would react to that feedback (I was generally surprised at how many people got defensive about fairly objective observations/questions). Overall I think it was quite a good data point. It was nice to have a more low-key and extensive complement to the more stressful live whiteboard interview (yes, we did have one of those, and I still think it was useful).
- fsk 11y ago
  When you get a coding assignment, you have no idea if they're going to give honest feedback if you submit a solution, or if they're just going to say "Thanks for applying, but we decided to go in a different direction."
- hadees 11y ago
  If you gave me a coding assignment that would take a day to complete I just wouldn't do it and move on to the next company. I want to work for a company that respects my time and knows that my code is worth money. A couple hours is all I would ever do.
MadManE 11y ago
I agree. I had an interview ask this, and it was essentially solving a problem that one of their clients had presented to them. And they needed it within a week. It felt very much like they were just exploiting applicants for crowd-sourced ideas.
rqebmm 11y ago
I often see complaints on HN about the silliness of whiteboarding in programming interviews, would you whiteboarding to a take-home assignment? Are there constraints you would prefer on one or the other? What amount of time spent qualifying for a single job is too much? I personally pushed to make our interview process more centered around a take-home style assignment, as I feel it's a much better approximation of programming skills than an in-person, high stress whiteboarding session, or a quiz on some programming trivia. We try to keep the assignment tightly constrained ("Here's a broken app with x requirements. Fix it.") in order to keep it to a few hours max. This I feel has been tremendously helpful for us, as I've never seen someone pass the take-home assignment and subsequently do poorly in the technical portion of our interview. Previously we had many people scrub out of our (admittedly pretty easy) technical interviews. Obviously asking for the take-home has resulted in a few false negatives, but I'd much rather an overly strong pre-interview filter to wasting hours of my entire team's time on a crappy interview.
on_and_off 11y ago
we do this at my current company. I think it is a great alternative to whiteboard tests as long as it is reasonably short. For mobile devs, we ask them to write a simple app solving a problem they should be familiar with and that we have had to deal with in the past.
mseebach 11y ago
I'm sure it's true that a non-trivial number of good candidates will reject a job opportunity when asked to do a work sample. These, however, will generally be those with plenty of attractive alternative offers - the ones well known in the community, with a good network etc. At the other end of the stick, there are those still trying to break in to the field. They may have moved to a new city, they may have worked in a totally unknown firm, they may have taken an "alternative" path through the educational system (and just to be blatantly obvious, the above describes me, circa 5 years ago) - their CVs don't jump out at employers, and as even phone screens are fairly expensive in engineer time, they are not very likely to be given a chance. Online coding challenges and take-home tests etc are a quick and cheap way to extend an opportunity to more "risky" candidates. I think that's an important element. (FTR, the company I ended up with, I got through to primarily via a "Who's hiring" thread on HN, however they did operate a "take home" test that took one hour).
- ruswick 11y ago
  It sounds like your process went ok. But one hour tests are one thing. Projects that may take 10 or more hours are totally different. It really is amazing what some of these companies will demand of their applicants before even granting an interview. And although this system might be more accessible than traditional recruiting, it is far from perfect. It is the job of the community to demand better and more respectful hiring practices from companies. Paying the applicant for the time they spent working on the project or offering a traditional interview for people who work full time and have families would be a good start.
russelluresti 11y ago
I think the take-home project can be successful, but not as most companies implement it. A basic rule of courtesy is that a company shouldn't ask the candidate to invest more time in the process than they are willing to invest. Most companies will spend an hour or half an hour talking to you on the phone and then give you a project that takes at least 8 hours to do. That's a ridiculous ask. It assumes that the candidate already knows they want to work for your company and is willing to invest that much time to get the job. They forget that an interview is a 2-way street. The candidate is also deciding if the company is the right fit for them, and being asked to invest a ton of your own time when the company doesn't seem willing to invest theirs is a huge turn off. There are a lot of advantages to a take-away task - such as being able to write actual code with a computer on your own stack as opposed to just writing on a white board (which some people have problems doing). However, this type of ask should only come after both the company and the individual have already invested significant and equal amounts of time - enough time for the candidate to be sure (or fairly confident) the company is a place they'd like to work. This means the take-away may only be useful after the on-site, or possibly right before it but after a few rounds where the candidate gets to understand the company better.
darklajid 11y ago
Spending that much time to qualify for a single job is too much to ask of anyone How can you state that as something absolute? I certainly understand your position, but I would be _glad_ to do something like that. My reasons - I hate white boards and quizzes. I would fail those, probably. - I'm not interested in travel time, just because. Give me 'homework' and only meet if you're prepared to make me an offer (other issues notwithstanding - it's okay if you reject me afterwards if I show up without wearing pants) without any more bullshit - I value my free time, but I also love to work on Things™. Any such 'homework' assignment would be a tiny pet project to drag me away from YT/HN/Awesomenauts and force me to learn something that I might not have picked myself In fact, I even thought about posting an Ask HN sort of thing to offer (limited somehow, day-job and being a dad and all) working for free in exchange for the input. I just haven't done it so far, because of fear of failure/rejection. So - you state a good point: People might not want to invest the time. But please speak for yourself only, not for the industry. I very much like the idea of presenting work done at home. That said, I probably wouldn't do both. If someone asks me quizzes first (and I manage to pass for weird reasons, stars align) I wouldn't invest more hours. You, the employer, just wasted hours with crap while you could've gotten ~reasonable~ accurate ideas about my work. If someone asks me to do one of these assignments, invites me to come over and THEN starts the quiz BS I'd probably get up and leave. Employers: Pick one way to test my skills. My preference: Judge skills in a remote/async process, use the face time to check for a cultural fit / negotiate / talk about the company, position and job: "We invited you after seeing your work, here's why you should work with us, this is the offer and these would be your coworkers - let's chat and do the tour"

odonnellryan 11y ago

I think it depends on the project. I'm looking to switch roles, and some projects are very small: solve a simple algo question. Should take less than an hour, which is less than the average interview, anyway! So no problem. Other projects are enormous. 40-80 HRs. of work, easily (write an entire web-app!) I did one of these... did not get the offer. I would not want to complete another.

eitally 11y ago

My wife is in this position now, and it's not for a programming job. She works in pharmaceuticals, in pharmacovigilance & risk management. She does a lot of data analysis & writing (the latest report she wrote for Health Canada & the FDA was 461 pages). A company she's interviewing with asked her to do a sample writing task as part of her application/interview. She hasn't decided whether it's worth it or not. For someone who has 15 years experience in the field, excellent recommendations and credentials, spending (her estimate) 12-15 hours doing pro bono work just to apply somewhere is ridiculous.

tlrobinson 11y ago
12+ hours is excessive even if you don't have 15 years experience.
akg_67 11y ago
I work in data analysis and I decline any "test" requests for a job or project. If you don't value my time before hiring, you definitely are not going to value my time after hiring. Also, if you are incapable of judging a fit based on resume, interviews, and just talking with me, you are not mature enough, experienced enough, smart enough and intelligent enough and have failed the test for being my potential coworker and boss. Edit: The tests make perfect sense for entry level positions and entry level candidates but once someone requesting tests for experienced positions and candidates, it shows the immaturity of the company, group, team and people who are unable to assess suitability of candidate and place too much emphasis on technical skills and tests as a crutch. The best jobs in my career were those where coworkers and boss had good relationship dynamics. Technical excellence had nothing to do with how well we worked and delivered results.
- Buetol 11y ago
  A simple solution is for the company to offer you something for the time you spent doing the sample test (money, gift card, their product,...)
  - brudgers 11y ago
    How about a car? Then it is obviously worth the applicant's time and since a professional can deliver or destroy a Bimmer's worth of corporate value on any given decision it easily pays for itself in identifying good candidates and weeding out bad ones.
    - vorg 11y ago
      > it easily pays for itself in identifying good candidates and weeding out bad ones It even easily pays for itself in just weeding out the bad ones.
technofiend 11y ago
I had a similar experience last year applying for a major technology company. Although I still lightly code in the devops sense (process automation) and to do typical management data analysis my day job is managing people, processes and crises. If you have billion dollar trading systems and need a cool-headed guy who can talk to VPs or directors in one breath and SAs, DBAs, SAN, Network, etc guys in the next I'm your man. I applied to head a global support organization, with teams in (from memory) North America, India and Asia Pacific. I'm pretty sure day-to-day would be a mix of employee career management, forward planning with my application development partners to make sure we were prepared capacity and training-wise to take on whatever's next, and crisis management of production issues and the like. Business as usual as they say. Yet I found myself answering computational complexity questions about hash map insertions. To be fair, before I became quite so management-centric I've found plenty of badly performing code via the usual tools (dtrace, strace, thread dumps), and sql queries using analogous database tools, but at some point you have to rely on technical people to do their jobs. Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? Or am I clueless?
- saidajigumi 11y ago
  > Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? An organization that does this is hiring stupidly. They have clearly demonstrated via interview that they don't understand the job to be done and/or how to determine that a candidate can do that job. Asking low-level technical questions is a terrible way to answer what I'll guess was the intended "question", something like: Does this candidate understand the technology and work well enough to make executive decisions grounded in our organization's reality?
- Eridrus 11y ago
  > at some point you have to rely on technical people to do their jobs. On one hand, I totally agree that technical tasks should be left to the technical people, preferably those who are actually working on the system. There's almost nothing worse than a manager trying to take technical decisions out of people's hands. > Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? Or am I clueless? On the other, I think people want to work for people they respect and to the extent that people respect technical chops that might be what they are screening for. Though the hash map question sounds like a soft ball, so it might be screening for a baseline of software competence.
IndianAstronaut 11y ago
>spending (her estimate) 12-15 hours doing pro bono work just to apply somewhere is ridiculous. Keep in mind that in today's job market, there are probably 10 people lined up who are willing to do just that.
- shredprez 11y ago
  Ah, but the willing aren't able and the able aren't willing. In any case, I'd avoid companies that recklessly fleece applicants just because they can. Once you've proven your worth professionally, begging for scraps is pathetic in any job market. For every greedy business with ten desperate applicants, there's an honest organization or untapped market ready to pay fairly for what value you offer. The question I consider when asked to perform for free: why should anyone simply donate their knowledge to a profitable business after spending a lifetime to attain it?
- ankitml 11y ago
  Doing something because you can is the worst reason to do so.
rwallace 11y ago
Does she have existing samples of writing that she can point to, not locked up under copyright? If so, then I would point to those samples and suggest those should be relevant data for assessment and I'd like to get further in the process before putting in extra work for this. If they're prepared to discuss the matter, great. If they just give a blank 'sorry, this is our policy,' then walk away. Why? Because if they're not willing to have a discussion with you, that means they don't care about you. It means they think they have a hefty surplus of candidates. Which means you are very unlikely to be the chosen one, and you would probably just waste a lot of your time.

ammon 11y ago

Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview. I think this might help a lot of people (clearly we have to test this)

rezistik 11y ago
I personally prefer it to a point. My only request when I get this type of interview is that I'm allowed to make the project and requirements public on my github for future interviewers. If I'm doing a free project I feel it's fair.
- ammon 11y ago
  Well, that's hard. Standardizing evaluation (critical to reducing bias) requires that everyone (or a lot of people) see a similar problem. And when someone is working on their own time, I think it's important that there's not working code easily accessible online. Perhaps this can be overcome by asking candidates to talk through their code after they are done
  - hayksaakian 11y ago
    if you change the assessment every so often, you could permit people to release their work eventually.
    - ammon 11y ago
      I like that idea!
    - Silhouette 11y ago
      If someone solved a significant problem at your request, but despite this you did not hire them or otherwise compensate them for that work, I don't see how you have any right either legally or ethically to expect them not to then use the work to advertise what they can do to anyone else. The premise is one-sided enough already (though the argument elsewhere in this HN discussion that it might be preferred by nervous interviewees is an interesting one I haven't noticed before) and I think expecting any sort of loyalty or confidentiality after the process has ended is rather optimistic.

Tech1 11y ago

I'm kind of in a similar boat. I've done them for companies that I like in the past, however I flat out refuse to do them for recruiting / job placement companies. ammon / uptown, I've put off doing my screen-sharing section of your interview process (I got pushed past the phone screen?) due to a lack of time. Maybe adding a way to 'schedule' it would be helpful for those of us that live by our g.calendars....and you all as well.

Harj 11y ago
What would be the best experience for you to schedule the interview? We'd definitely like to make that easier.
- Tech1 11y ago
  Just something simple, allow me to add it to my google calendar.

geebee 11y ago

I did this once. After a take home quiz that took about an hour, I had a 6-7 hour "homework" problem. I did it, and sent it in. It took the company a month to get back to me, though I did stay in contact with the recruiter. The final answer, delivered by the recruiter, was "we've decided not to more forward at this time." No other feedback. I have no idea if anyone even looked at it - I never did speak to a dev at the company. This experience left me embarrassed, honestly. I don't think it reflects well on me that I put up with it even once. At this point, I will participate in tech interviews, but I won't do any more take-home assignments. I suppose this could change under different circumstances. Gayle Laakmann McDowell, who wrote "Cracking the Coding Interview", wrote an insightful post her blog about this. It's titled "Companies who give candidates homework assignments: knock it off!" http://www.gayle.com/blog/2013/09/18/companies-who-give-candidates-homework-assignments-knock-it-off http://www.gayle.com/blog/2013/09/18/companies-who-give-cand... She mentions something I see as a problem as well - these tests allow the company to burn a lot of a candidate's time, but not the other way around. I may have burned 6+ hours interviewing unsuccessfully at google, but they parted with 6+ hours of developer time as well (though I did spend a lot of additional hours reviewing algorithms and doing sample problems). This creates a natural set of checks and balances that don't exist with homework assignments. She does have some good suggestions about how to use a take home if a company is determined to do it. To me, a particularly insightful one was to have a high (she says 90%+) passage rate. I agree. If you're going to ask a candidate to spend a day on a homework assignment, you should be very close to an offer. If you're using it as an early filter, you are wasting a tremendous amount of developer time.

jackieDQ 11y ago
Yep I've been burned a couple times by this and only under certain circumstances will I do a take home project. Even worse is when they also ask before hand not to post it to your github account. If they aren't going to give me feedback, then I'm going to host it and seek it out elsewhere.
- solve 11y ago
  Love that. If they burn too many people, they have to create new questions.

techiferous 11y ago

I've had a different experience on the hiring side of the table. The coding exercise that we provide as part of the interview has been very successful in surfacing the skill and talent of the programmer. By using that screening device, we have assembled a team of very sharp programmers. It's totally worth filtering out people who choose not to spend the time. Even though we end up losing some potential candidates, it's worth it for a 0% chance of a mishire for technical reasons.

solve 11y ago
A take-home test has a 0% chance of being cheated on? Are you sure you're talking about the same thing here?
- techiferous 11y ago
  Our exercise can't be cheated on because we discuss it with the candidate during the in-person interview and even ask them to change the code in a simple way. Also, the exercise is designed to be original and so a solution can't be Googled.
  - solve 11y ago
    We're talking only about the "in their own time" take-home types of tests. The type of cheating I'm talking about is outsourcing the problem or even just having a friend do it for them.
jrs235 11y ago
We're currently hiring and for the first time, we should have started doing this years ago, given a simple coding exercise. I estimate it would/should take any programmer/problem solver no more than 15, maybe 20, minutes to come up with the solution if they have experience in the language we request they solve it in. The exercise doesn't require specific domain knowledge but should quickly assess whether a programmer, particularly with experience in a different language, is able to learn and apply the syntax and semantics of another language. If they are completely unfamiliar with the language then maybe an hour. They can use whatever references they want. We will check for complete copy/pasted code and alter the requirements and/or test data over time. The other great benefit is we get to see what/if/how they ask questions to clarify the problem and solution. EDIT: This exercise is geared towards fresh graduates or developers that don't have experience in the language we use. Can it be gamed? Yes. Might some people not bother? Yes. Are we okay with that? Yes.

solve 11y ago

My rule for these - only do these tests in-person, on-site, and after after the company has invested a significant amount of their employees' time to interview you in-person. Tons of companies hand out these "homework" tests to hundreds of people without having even the slightest intention of hiring them. F* that.

s0uthPaw88 11y ago

I feel that if you are asking a company to invest $100k/yr on you, it is not unreasonable for them to ask for a few hours of your time.

ruswick 11y ago
The point is that they pay you the $100k for the time you give them as an employee. They don't pay you a cent for the interview project, so you shouldn't give them a single second in return.
- rqebmm 11y ago
  They don't pay you for the interview, but that usually costs the company several man-hours worth of work just for the time an applicant is on-site. Should applicants be reimbursing companies for failed interviews?
  - ruswick 11y ago
    Maybe. I've always thought it would be interesting to make applicants pay an application fee. This would cut down on people "spraying and praying" with their application, and would lessen the workload for companies. It would also justify spending more time and effort in reading applications, since the company isn't just wasting resources reading bad applications. Even under the current system, at least the waste is symmetric. I give up a few hours of my time, and the company gives up a few hours of its time. There's equity. The mutual work that the company and applicant do offset each other. A take-home test model skews that balance in favor of the company. An applicant can spend 10 or more hours working the project. The company can run it through an automated testing suite and have a recruiter spend five minutes looking it over. The system is designed to waste more of the applicant's time and less of the company's.
    - rqebmm 11y ago
      > Even under the current system, at least the waste is symmetric. But it's never symmetric. Someone in HR spent time setting up the job posting and managing the process. Someone in management took the time to respond to HR and review your resum, then approve the interview. At least one person at a time is in the interview with you. Then the entire team will spend time afterward breaking it down. A single hour spent by a bad candidate wastes at least 3 man-hours of work by the company, and most likely more.

ruswick 11y ago

Also keep in mind that applying to jobs is frequently throwing your application into a black hole and just praying. Sending a resume into the black hole is one thing. Sending a large codebase or project you worked on as part of the application is entirely different. I remember a few months ago, I was applying for an internship with a company and they requested that I write a web app to test coding proficiency before they thought about granting an interview. I spent about 12-15 hours writing the thing, deployed the app to Heroku and put the code on Github. I sent a link to the app and the repo to the recruiter. I never got a response of any kind. Although some companies do try to make the application experience as pleasant as possible for the applicant, the majority of companies don't give a shit about the people who want to work for them, and make no attempt to respect the time of their applicants. Placing greater demands on the applicant is an easy way to shift more of the workload from the recruiter to the applicant. The applicant is the loser in this situation.

dfan 11y ago

My last company had a screening coding problem that was designed to take around a day (although it didn't have to; when I did it as a reference point, it took a couple of hours). I was always kind of astonished how many applicants were willing to do the whole thing, given, as you say, that they often already had a full-time job and/or were applying for multiple positions. We always gave really extensive feedback on it, partially because one of the things were were looking for was how people would react to that feedback (I was generally surprised at how many people got defensive about fairly objective observations/questions). Overall I think it was quite a good data point. It was nice to have a more low-key and extensive complement to the more stressful live whiteboard interview (yes, we did have one of those, and I still think it was useful).

fsk 11y ago
When you get a coding assignment, you have no idea if they're going to give honest feedback if you submit a solution, or if they're just going to say "Thanks for applying, but we decided to go in a different direction."
hadees 11y ago
If you gave me a coding assignment that would take a day to complete I just wouldn't do it and move on to the next company. I want to work for a company that respects my time and knows that my code is worth money. A couple hours is all I would ever do.

MadManE 11y ago

I agree. I had an interview ask this, and it was essentially solving a problem that one of their clients had presented to them. And they needed it within a week. It felt very much like they were just exploiting applicants for crowd-sourced ideas.

rqebmm 11y ago

I often see complaints on HN about the silliness of whiteboarding in programming interviews, would you whiteboarding to a take-home assignment? Are there constraints you would prefer on one or the other? What amount of time spent qualifying for a single job is too much? I personally pushed to make our interview process more centered around a take-home style assignment, as I feel it's a much better approximation of programming skills than an in-person, high stress whiteboarding session, or a quiz on some programming trivia. We try to keep the assignment tightly constrained ("Here's a broken app with x requirements. Fix it.") in order to keep it to a few hours max. This I feel has been tremendously helpful for us, as I've never seen someone pass the take-home assignment and subsequently do poorly in the technical portion of our interview. Previously we had many people scrub out of our (admittedly pretty easy) technical interviews. Obviously asking for the take-home has resulted in a few false negatives, but I'd much rather an overly strong pre-interview filter to wasting hours of my entire team's time on a crappy interview.

on_and_off 11y ago

we do this at my current company. I think it is a great alternative to whiteboard tests as long as it is reasonably short. For mobile devs, we ask them to write a simple app solving a problem they should be familiar with and that we have had to deal with in the past.

mseebach 11y ago

I'm sure it's true that a non-trivial number of good candidates will reject a job opportunity when asked to do a work sample. These, however, will generally be those with plenty of attractive alternative offers - the ones well known in the community, with a good network etc. At the other end of the stick, there are those still trying to break in to the field. They may have moved to a new city, they may have worked in a totally unknown firm, they may have taken an "alternative" path through the educational system (and just to be blatantly obvious, the above describes me, circa 5 years ago) - their CVs don't jump out at employers, and as even phone screens are fairly expensive in engineer time, they are not very likely to be given a chance. Online coding challenges and take-home tests etc are a quick and cheap way to extend an opportunity to more "risky" candidates. I think that's an important element. (FTR, the company I ended up with, I got through to primarily via a "Who's hiring" thread on HN, however they did operate a "take home" test that took one hour).

ruswick 11y ago
It sounds like your process went ok. But one hour tests are one thing. Projects that may take 10 or more hours are totally different. It really is amazing what some of these companies will demand of their applicants before even granting an interview. And although this system might be more accessible than traditional recruiting, it is far from perfect. It is the job of the community to demand better and more respectful hiring practices from companies. Paying the applicant for the time they spent working on the project or offering a traditional interview for people who work full time and have families would be a good start.

russelluresti 11y ago

I think the take-home project can be successful, but not as most companies implement it. A basic rule of courtesy is that a company shouldn't ask the candidate to invest more time in the process than they are willing to invest. Most companies will spend an hour or half an hour talking to you on the phone and then give you a project that takes at least 8 hours to do. That's a ridiculous ask. It assumes that the candidate already knows they want to work for your company and is willing to invest that much time to get the job. They forget that an interview is a 2-way street. The candidate is also deciding if the company is the right fit for them, and being asked to invest a ton of your own time when the company doesn't seem willing to invest theirs is a huge turn off. There are a lot of advantages to a take-away task - such as being able to write actual code with a computer on your own stack as opposed to just writing on a white board (which some people have problems doing). However, this type of ask should only come after both the company and the individual have already invested significant and equal amounts of time - enough time for the candidate to be sure (or fairly confident) the company is a place they'd like to work. This means the take-away may only be useful after the on-site, or possibly right before it but after a few rounds where the candidate gets to understand the company better.

darklajid 11y ago

Spending that much time to qualify for a single job is too much to ask of anyone How can you state that as something absolute? I certainly understand your position, but I would be _glad_ to do something like that. My reasons - I hate white boards and quizzes. I would fail those, probably. - I'm not interested in travel time, just because. Give me 'homework' and only meet if you're prepared to make me an offer (other issues notwithstanding - it's okay if you reject me afterwards if I show up without wearing pants) without any more bullshit - I value my free time, but I also love to work on Things™. Any such 'homework' assignment would be a tiny pet project to drag me away from YT/HN/Awesomenauts and force me to learn something that I might not have picked myself In fact, I even thought about posting an Ask HN sort of thing to offer (limited somehow, day-job and being a dad and all) working for free in exchange for the input. I just haven't done it so far, because of fear of failure/rejection. So - you state a good point: People might not want to invest the time. But please speak for yourself only, not for the industry. I very much like the idea of presenting work done at home. That said, I probably wouldn't do both. If someone asks me quizzes first (and I manage to pass for weird reasons, stars align) I wouldn't invest more hours. You, the employer, just wasted hours with crap while you could've gotten ~reasonable~ accurate ideas about my work. If someone asks me to do one of these assignments, invites me to come over and THEN starts the quiz BS I'd probably get up and leave. Employers: Pick one way to test my skills. My preference: Judge skills in a remote/async process, use the face time to check for a cultural fit / negotiate / talk about the company, position and job: "We invited you after seeing your work, here's why you should work with us, this is the offer and these would be your coworkers - let's chat and do the tour"