6 ms·

Your experience sounds horrible, and unfortunately those sort of "tests" are just a horrible thing. Generally, do NOT do take-home exercises. That's your time,

by kayo_20211030 4y ago

Your experience sounds horrible, and unfortunately those sort of "tests" are just a horrible thing. Generally, do NOT do take-home exercises. That's your time, which has value; the company doing the hiring contributes nothing. At least, in a live-coding exercise both you and they have some skin in the game. As with all tests, good or bad, part of the test is actually doing the test. Often it's less what you know, or could do; and more about "can you do the test?", and mostly "doing the test" is a poor proxy, and the whole exercise becomes a waste of everyone's time.

garrettdimon 4y ago
That’s not always accurate that the company doing the hiring contributes nothing. (Also good companies could offer to pay for that take-home test time.) Having done take home before from the hiring side, it was incredibly time-consuming for us. We had someone anonymize the three finalist submissions, and then we had three people each individually review and comment on each one, and then we got together to discuss and choose the final candidate. Once we agreed on one, only then was it de-anonymized. All-in, it took way more time than a single developer doing three live coding interviews. But my guess would be that most companies wouldn’t be willing to be that deliberate with take home.
- emodendroket 4y ago
  There’s no guarantee the employer isn’t in final phases with someone else that will lead to them never even evaluating your work.
mannyv 4y ago
The last time I did a take home exercise for a position I wrote something that was beyond the team's skillset. That was awkward.
- sjducb 4y ago
  That's a great reason to do the take home test. You learn a huge amount about the team by how they react to your code.
  - emodendroket 4y ago
    Or they just evaluate among themselves and never tell you what they thought beyond thumbs up or thumbs down.
    - RoyGBivCap 4y ago
      Or just completely drop the ball and never get back to you.
decafninja 4y ago
There's also the issue of scalability. Grinding leetcode at least scales horizontally to a huge number of companies. Homework is typically useless outside of the single company you're doing it for. I generally refuse all takehome assignments unless: 1.) It sounds uniquely interesting and fun to do. 2.) The company is prestigious enough, or pays well enough, that making any effort to try to get the job worth it.
- emodendroket 4y ago
  Completely agreed. Also in my experience the takehomes are even more susceptible to being rejected for absolute bullshit reasons. Someone told me the thing I coded up in less than an hour to solve their problem was “overengineered” so they wouldn’t be moving forward.
  - decafninja 4y ago
    Also... I think at 100% of the companies that have given me a homework assignment, it has always been in addition to, not in place of, a standard leetcode whiteboard round.
    - emodendroket 4y ago
      Same here. Though it’s hard to blame them since some people quite obviously didn’t do the takehomes themselves.
- whynotminot 4y ago
  Why would you be interviewing at all if the criteria for (2) isn’t met? Unless you’re just trying to get interview practice or keeping them as a safety option.
  - kps 4y ago
    Sometimes people just need a job (especially in the US, where jobs are so coupled to health care). I've been in that position, and I'm very glad I'm not now.
eesmith 4y ago
The author did write "better yet, let applicants do a take-home, and pay them for their time. ... think of it as another chance to be selling them on joining your team over some other team."
- garrettdimon 4y ago
  I added that after-the-fact in response to this comment. So that's on me for not including it originally. It's a great idea and one I wholeheartedly endorse because it forces the company to put some skin in the game and helps limit the number of applicants that they would request perform the take home test while also recognizing that applicants' time is in limited supply.
viraptor 4y ago
> Often it's less what you know, or could do; and more about "can you do the test?" It really depends on how you create the test and how you review it. You can prepare one where the actual solution is 10 lines of code and anything else is exactly for showing what the candidate knows / could do. I used something like "read data from CSV, write it to sqlite, treat it like a mature production app, feel free to use placeholders (usage doc goes here), go nuts". Then the test was really about knowing about error recovery, encoding issues, documentation, error reporting, monitoring, ci pipelines, etc. Many badly organised tests don't make the whole concept of a take home test bad.
- frumper 4y ago
  Your test sounds like what people complain about. You give them instructions but it's "really about ... ". Just ask them what you want them to know. If you hired this person and they turned in code without error recovery or encoding, you'd tell them they need to add it and they would. No need to obfuscate your intentions and have them read your mind in what you're looking for. No take home test is even needed. A simple conversation would let you know if they understand the importance of those things.
  - viraptor 4y ago
    > Just ask them what you want them to know. You can't really ask about those things directly. If I ask "(how) would you add monitoring to this?" you can make something up on the spot. If I ask "what else would a mature production app contain?" that will give me a better idea of what you're familiar with. > you'd tell them they need to add it and they would Different levels. We were not after juniors in that case, but people who can do independent work. The question would be way more specific otherwise. > No need to obfuscate your intentions and have them read your mind in what you're looking for. It's not about mind reading. For that role, you either know how to create/manage serious applications or not. Your specific ideas may be different than mine and that's fine, but the general areas of interest will match. E.g. whether you register some event notifications, do text logging, persist decision journal, or do something else — you'll likely mention that. And you may forget/miss one or two things - that happens. But you don't miss them all if you're a good candidate for that position.
OkayPhysicist 4y ago
One of the better laid out interviews I had consisted of them pitching a problem prompt via email, followed by an hour and a half to complete the problem, followed by a ~30 minute code review. Realistically, the task was like a 20 minute job, so plenty of margin for stress-related slowdown. I think that's a happy medium between unbounded take home exams that monopolize your time, versus in-call leetcode whiteboard sessions.
akvadrako 4y ago
Take home tests are often paid and those are the only ones I will do anymore after being burned a few times. That solves the skin in the game problem.
- avnigo 4y ago
  If they're not paid and you still wanna do it, make it into an interesting problem you wanna work on and have it be presentable enough to include in your portfolio. At least, that way you might get something out of it instead of nothing, although I definitely do agree that being paid for it is the only way to solve the skin in the game problem.
throwaway675309 4y ago
I think we're going to see a pretty large industry shift away from take-home tests given how easily 90% of these tests can be gamed using chat GPT.
- ravenstine 4y ago
  I'd think we'd have already seen this with GitHub Copilot. There was an interview I was part of late last year where the candidate had Copilot turned on during the live coding, and he didn't turn it off even after it was obvious that's what he was using. What I was more surprised by is how I thought this was a bigger deal than everyone else. Like, why come up with these elaborate tests when the candidate is just going to use autocomplete the whole time? Maybe programming in another few years will just be glorified autocomplete and little more. And perhaps testing people on how to write code was a mistake to begin with. It's one thing to write code, but reading code is another.
  - 8note 4y ago
    If they are effective at using it to solve problems, why not? The real challenge is to your testing mechanism and not the candidate
  - d0mine 4y ago
    > Maybe programming in another few years will just be glorified autocomplete and little more. It won't. For the same reason code generation from UML diagrams hasn't replaced programmers [1]. Or why business owner/analysts don't write acceptance tests in Gherkin (Given-When-Then). Though some tasks that programmers had to do manually in the past may be automated (not the first time). [1]: https://news.ycombinator.com/item?id=26934795 https://news.ycombinator.com/item?id=26934795

garrettdimon 4y ago

That’s not always accurate that the company doing the hiring contributes nothing. (Also good companies could offer to pay for that take-home test time.) Having done take home before from the hiring side, it was incredibly time-consuming for us. We had someone anonymize the three finalist submissions, and then we had three people each individually review and comment on each one, and then we got together to discuss and choose the final candidate. Once we agreed on one, only then was it de-anonymized. All-in, it took way more time than a single developer doing three live coding interviews. But my guess would be that most companies wouldn’t be willing to be that deliberate with take home.

emodendroket 4y ago
There’s no guarantee the employer isn’t in final phases with someone else that will lead to them never even evaluating your work.

mannyv 4y ago

The last time I did a take home exercise for a position I wrote something that was beyond the team's skillset. That was awkward.

sjducb 4y ago
That's a great reason to do the take home test. You learn a huge amount about the team by how they react to your code.
- emodendroket 4y ago
  Or they just evaluate among themselves and never tell you what they thought beyond thumbs up or thumbs down.
  - RoyGBivCap 4y ago
    Or just completely drop the ball and never get back to you.

decafninja 4y ago

There's also the issue of scalability. Grinding leetcode at least scales horizontally to a huge number of companies. Homework is typically useless outside of the single company you're doing it for. I generally refuse all takehome assignments unless: 1.) It sounds uniquely interesting and fun to do. 2.) The company is prestigious enough, or pays well enough, that making any effort to try to get the job worth it.

emodendroket 4y ago
Completely agreed. Also in my experience the takehomes are even more susceptible to being rejected for absolute bullshit reasons. Someone told me the thing I coded up in less than an hour to solve their problem was “overengineered” so they wouldn’t be moving forward.
- decafninja 4y ago
  Also... I think at 100% of the companies that have given me a homework assignment, it has always been in addition to, not in place of, a standard leetcode whiteboard round.
  - emodendroket 4y ago
    Same here. Though it’s hard to blame them since some people quite obviously didn’t do the takehomes themselves.
whynotminot 4y ago
Why would you be interviewing at all if the criteria for (2) isn’t met? Unless you’re just trying to get interview practice or keeping them as a safety option.
- kps 4y ago
  Sometimes people just need a job (especially in the US, where jobs are so coupled to health care). I've been in that position, and I'm very glad I'm not now.

eesmith 4y ago

The author did write "better yet, let applicants do a take-home, and pay them for their time. ... think of it as another chance to be selling them on joining your team over some other team."

garrettdimon 4y ago
I added that after-the-fact in response to this comment. So that's on me for not including it originally. It's a great idea and one I wholeheartedly endorse because it forces the company to put some skin in the game and helps limit the number of applicants that they would request perform the take home test while also recognizing that applicants' time is in limited supply.

viraptor 4y ago

> Often it's less what you know, or could do; and more about "can you do the test?" It really depends on how you create the test and how you review it. You can prepare one where the actual solution is 10 lines of code and anything else is exactly for showing what the candidate knows / could do. I used something like "read data from CSV, write it to sqlite, treat it like a mature production app, feel free to use placeholders (usage doc goes here), go nuts". Then the test was really about knowing about error recovery, encoding issues, documentation, error reporting, monitoring, ci pipelines, etc. Many badly organised tests don't make the whole concept of a take home test bad.

frumper 4y ago
Your test sounds like what people complain about. You give them instructions but it's "really about ... ". Just ask them what you want them to know. If you hired this person and they turned in code without error recovery or encoding, you'd tell them they need to add it and they would. No need to obfuscate your intentions and have them read your mind in what you're looking for. No take home test is even needed. A simple conversation would let you know if they understand the importance of those things.
- viraptor 4y ago
  > Just ask them what you want them to know. You can't really ask about those things directly. If I ask "(how) would you add monitoring to this?" you can make something up on the spot. If I ask "what else would a mature production app contain?" that will give me a better idea of what you're familiar with. > you'd tell them they need to add it and they would Different levels. We were not after juniors in that case, but people who can do independent work. The question would be way more specific otherwise. > No need to obfuscate your intentions and have them read your mind in what you're looking for. It's not about mind reading. For that role, you either know how to create/manage serious applications or not. Your specific ideas may be different than mine and that's fine, but the general areas of interest will match. E.g. whether you register some event notifications, do text logging, persist decision journal, or do something else — you'll likely mention that. And you may forget/miss one or two things - that happens. But you don't miss them all if you're a good candidate for that position.

OkayPhysicist 4y ago

One of the better laid out interviews I had consisted of them pitching a problem prompt via email, followed by an hour and a half to complete the problem, followed by a ~30 minute code review. Realistically, the task was like a 20 minute job, so plenty of margin for stress-related slowdown. I think that's a happy medium between unbounded take home exams that monopolize your time, versus in-call leetcode whiteboard sessions.

akvadrako 4y ago

Take home tests are often paid and those are the only ones I will do anymore after being burned a few times. That solves the skin in the game problem.

avnigo 4y ago
If they're not paid and you still wanna do it, make it into an interesting problem you wanna work on and have it be presentable enough to include in your portfolio. At least, that way you might get something out of it instead of nothing, although I definitely do agree that being paid for it is the only way to solve the skin in the game problem.

throwaway675309 4y ago

I think we're going to see a pretty large industry shift away from take-home tests given how easily 90% of these tests can be gamed using chat GPT.

ravenstine 4y ago
I'd think we'd have already seen this with GitHub Copilot. There was an interview I was part of late last year where the candidate had Copilot turned on during the live coding, and he didn't turn it off even after it was obvious that's what he was using. What I was more surprised by is how I thought this was a bigger deal than everyone else. Like, why come up with these elaborate tests when the candidate is just going to use autocomplete the whole time? Maybe programming in another few years will just be glorified autocomplete and little more. And perhaps testing people on how to write code was a mistake to begin with. It's one thing to write code, but reading code is another.
- 8note 4y ago
  If they are effective at using it to solve problems, why not? The real challenge is to your testing mechanism and not the candidate
- d0mine 4y ago
  > Maybe programming in another few years will just be glorified autocomplete and little more. It won't. For the same reason code generation from UML diagrams hasn't replaced programmers [1]. Or why business owner/analysts don't write acceptance tests in Gherkin (Given-When-Then). Though some tasks that programmers had to do manually in the past may be automated (not the first time). [1]: https://news.ycombinator.com/item?id=26934795 https://news.ycombinator.com/item?id=26934795