5 ms·

Take home coding tests are fine. I really prefer them, honestly. The problem is when you do the test, write great code, they love it and ask you to come onsite,

Take home coding tests are fine. I really prefer them, honestly. The problem is when you do the test, write great code, they love it and ask you to come onsite, and then you get white-boarded by a group of people who never even looked at your code or resume and get rejected for not being able to cough up a DFS algorithm on the spot.

ritchiea 7y ago
I agree with you and there is nothing inherently wrong about a take home test. I do prefer taking home a test and working on a feature and talking about it with the hiring panel over whiteboarding. But the expectations they express about the take home work can still be poorly thought out and make me feel like I'm in the wrong for taking a longer time than they budgeted for me. Or that I'm being secretly tested to see if I will ignore the suggestion and throw myself at this exam to show my dedication to their company. This joke is a suggestion there is still room for improvement.
- ritchiea 7y ago
  Oh and I will add, I hate the criticism that a take home test is "free, unpaid work." So is looking for jobs and taking time out of your schedule to interview on site. It is inevitable that in a job search you put in unpaid time to try to find the best possible work experience. But it would be nice to try to find ways to make it less painful for job seekers and more accurate in assessing developers's real ability to be productive team members.
  - rtkwe 7y ago
    The distinction between the two though is projects and coding tests are work actually potentially useful to the company, things they could take and use where what you're talking about isn't part of the normal work of an employee.
    - mooreds 7y ago
      I've only done a few take home tests, but never done one (or given one) where the work product was usable for anything other than evaluating candidate skills.
    - hellisothers 7y ago
      This seems like some sort of myth that happened to one engineer and now everybody is repeating it. Take homes I’ve done are no more useful to the company than any of the 1hr whiteboard problems. The company is spending time and money (a LOT of money at big companies) to interview you, expect to spend some of your own.
      - subhobroto 7y ago
        > This seems like some sort of myth that happened to one engineer and now everybody is repeating it. Take homes I’ve done are no more useful to the company than any of the 1hr whiteboard problems You underestimate the insight that comes with looking at different solutions to the same problem. I have been showing the value of GraphQL to a few startups for a while now to the point where I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription. Create new users, tasks, updates, votes? Check Check Update users, tasks, updates, votes? Check Get realtime notification of changes to users, tasks, updates, votes? Check This new insight has tremendous value to a company that's struggling with a RPC like system hidden behind "REST" endpoints. At a company I know the R&D team routinely interviews PhD candidates of which an hour is kept aside specifically for the R&D team to come up to speed on the latest research that these PhD candidates read up on. > The company is spending time and money (a LOT of money at big companies) to interview you, expect to spend some of your own. I dont think you thought this through. The company is spending time and money to make sure they don't make a hiring mistake. The time and effort sunk in by the candidate is part of the cost subsidized by the candidate. An argument can be made that it would be cheaper for the employer to hire a candidate on an at-will basis once the candidate passes a basic skill and design test.
  - 52-6F-62 7y ago
    I've repeated this several times now but a great experience I had was with a small company from Ottawa (they may be reading this). They offered to pay me for time spent working asynchronously on a task with them that they would potentially be able to use. They gave me a retainer up front and had me bill them based on agreed-upon hourly amount. Ultimately the offer was based on that work. We ended up pressing pause as they were able to find help more local to them so I didn't join up but all around good experience. And it is certainly nice to be paid when it is work they may potentially use!
    - subhobroto 7y ago
      > they were able to find help more local to them so I didn't join up but all around good experience Dodged a bullet there. If there's a preference for a co-located candidate, that means their processes are not built to put a remote candidate on equal footing. Remote work is hard. NOTHING beats the bandwidth and fidelity of face to face conversations. As a result, remote workers need MORE help, not less to be successful. One way of MORE help is to have an async work flow. Let me guess: 1. this company has most of their employees live in the same city or state although they all "work from home most of the time"? 2. this company has a slack channel where their employees hang out and are almost always available?
      - 52-6F-62 7y ago
        In this case I think your assumptions were incorrect. They already had some remote staff and contacted me out of my expressed desire to work remotely. I was caught up in a critical project and needed more time however they needed the extra set of hands earlier than I could commit to. I'm guessing colocation was an added bonus because it is a bit easier. I think my largest concern was the sustainability of my rate for them in the long term being a small outfit. No regrets one way or the other. And like I said, they paid my retainer up front and it was definitely to my satisfaction.
        subhobroto 7y ago
        > I'm guessing colocation was an added bonus because it is a bit easier. I would love to hear your thoughts on this: My opinion is that if there's a preference for a co-located candidate, that simply means their processes are not built to put a remote candidate on equal footing. Hence the preference. The fact that they have remote workers does not invalidate the fact that those remote workers are at a disadvantage. > I think my largest concern was the sustainability of my rate for them in the long term being a small outfit This is one more red flag: companies that have to work remotely because they have to not because they want to. There is a world of difference between a company that had to settle for remote work just because they could not afford to hire locally from a company that was built remote first because the founders believed in remote work. The former will always be on the lookout for local candidates, forever deprioritizing remote workers while the latter don't have any such discrimination in mind. I am glad it turned out well for you in the short term. Maybe you can follow up on how their remote workers are currently doing.
nostrebored 7y ago
I'm always shocked by the examples that come up when people talk about when talking about whiteboarding. DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications.
- cblades 7y ago
  That there is a plethora of implementations of and documentation available for. There is no need to memorize an implementation.
- subhobroto 7y ago
  Sure, DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications. I have coded DFS many times in my life, both at high school and atleast a dozen times each during my Bachelors and Masters and far more times at work because I do a lot of NLP and traversing a tree is a regular day at work. I can code DFS with you today, when out at a bar over a few beers. What I struggle with is to code DFS: 1. with people I don't know, am not comfortable with 2. who are looking at me expectantly, expecting me to fail at it (because that's what happens in most of the interviews they have been part of) 3. with time running out They only time I was able to perform DFS at an interview was at my first job out of college because just the month before I had prepared for my Data Structure final exam and could code DFS (or any of the HackerRank style algorithms) in my sleep no matter what even if all I had was a chisel and a slab of granite. I was not upset that I never had to actually code the DFS, invert a binary tree and find the minimum coin change (all the 3 questions I was asked for the job) ever after that - it was all C++/Python/Java CRUD, for 3 years because I had 0 professional experience with CRUD at that point so I would have never got the job in the first place. In fact the official policy at that place was to always use existing commercial libraries (and purchase them if necessary) because it was a CRUD shop and they had high turnover so any custom library would quickly become a liability.
mmazing 7y ago
I've interviewed several hundred people and gave them a take home test of a simple problem that we had encountered in our work. We asked them to write a simple API that would return the closest pharmacy, given a latitude and longitude supplied to the API, and gave them a CSV with 10 pharmacies. Language didn't matter, and we never even ran it to see if it actually worked. During the interview we would have them walk us through the code and explain what each part does and why they did whatever (this makes sure they actually wrote it and understand it). Then, we would throw curveballs like "what would you need to change in your code if there were 100,000 pharmacies", and "how would you need to change it to return the closest 10 pharmacies instead of the closest 1?" It's normally really difficult to get into someone's head about HOW they solve problems. I really felt like our method shined a light on their programming abilities. Also, how well they might anticipate future usage by building something smart to begin with and not code themselves into a corner.
subhobroto 7y ago
> Take home coding tests are fine. I really prefer them, honestly Home coding tests are great at gauging real life performance of a candidate, if done right, but in my limited exposure (I don't have a lot of time to work on coding tests just so I can collect data samples to back my opinion), coding tests are handled extremely poorly, with expectations being set extremely vaguely. Any realistic assignment requires a dialogue between the consumer and producer. Companies are trying to minimize the time/money they spend on each interview. Clarifications cost money. Hoping a company will engage in a "what really are your requirements? are you trying to test my basic coding competency or do you want to see the best I have got?" will elicit a very vague response, if any. If a company treats a home coding test as a real assignment and provides the candidate with the resources they need, I see no problem. Unfortunately from my personal experience and anecdotes from others, it's just not the case. > get rejected for not being able to cough up a DFS algorithm on the spot DFS is easy. BFS is easy. So is Coin changing, inverting binary trees and doing back flips. If you invest enough time and effort into it, you can do it. I have done DFS, BFS, Coin changing, inverting binary trees many times before: https://news.ycombinator.com/item?id=22368645 https://news.ycombinator.com/item?id=22368645 I am beginning to prefer DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews. I have to learn it once and can regurgitate until I forget them. I pay the heavy cost once upfront and amortize it. No such thing for a home coding test! I now have to pay the same cost over and over again, unable to amortize it because no two companies are solving the same problem unless you view it as a meta problem and in that case you might very well end up redesigning Django, RoR or a similar meta framework. Financially it would be cheaper to become a full time contractor and get paid to make POCs and demos unencumbered by NDAs. As a candidate a home coding test is extremely expensive compared to the traditional DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews.

ritchiea 7y ago

I agree with you and there is nothing inherently wrong about a take home test. I do prefer taking home a test and working on a feature and talking about it with the hiring panel over whiteboarding. But the expectations they express about the take home work can still be poorly thought out and make me feel like I'm in the wrong for taking a longer time than they budgeted for me. Or that I'm being secretly tested to see if I will ignore the suggestion and throw myself at this exam to show my dedication to their company. This joke is a suggestion there is still room for improvement.

ritchiea 7y ago
Oh and I will add, I hate the criticism that a take home test is "free, unpaid work." So is looking for jobs and taking time out of your schedule to interview on site. It is inevitable that in a job search you put in unpaid time to try to find the best possible work experience. But it would be nice to try to find ways to make it less painful for job seekers and more accurate in assessing developers's real ability to be productive team members.
- rtkwe 7y ago
  The distinction between the two though is projects and coding tests are work actually potentially useful to the company, things they could take and use where what you're talking about isn't part of the normal work of an employee.
  - mooreds 7y ago
    I've only done a few take home tests, but never done one (or given one) where the work product was usable for anything other than evaluating candidate skills.
  - hellisothers 7y ago
    This seems like some sort of myth that happened to one engineer and now everybody is repeating it. Take homes I’ve done are no more useful to the company than any of the 1hr whiteboard problems. The company is spending time and money (a LOT of money at big companies) to interview you, expect to spend some of your own.
    - subhobroto 7y ago
      > This seems like some sort of myth that happened to one engineer and now everybody is repeating it. Take homes I’ve done are no more useful to the company than any of the 1hr whiteboard problems You underestimate the insight that comes with looking at different solutions to the same problem. I have been showing the value of GraphQL to a few startups for a while now to the point where I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription. Create new users, tasks, updates, votes? Check Check Update users, tasks, updates, votes? Check Get realtime notification of changes to users, tasks, updates, votes? Check This new insight has tremendous value to a company that's struggling with a RPC like system hidden behind "REST" endpoints. At a company I know the R&D team routinely interviews PhD candidates of which an hour is kept aside specifically for the R&D team to come up to speed on the latest research that these PhD candidates read up on. > The company is spending time and money (a LOT of money at big companies) to interview you, expect to spend some of your own. I dont think you thought this through. The company is spending time and money to make sure they don't make a hiring mistake. The time and effort sunk in by the candidate is part of the cost subsidized by the candidate. An argument can be made that it would be cheaper for the employer to hire a candidate on an at-will basis once the candidate passes a basic skill and design test.
- 52-6F-62 7y ago
  I've repeated this several times now but a great experience I had was with a small company from Ottawa (they may be reading this). They offered to pay me for time spent working asynchronously on a task with them that they would potentially be able to use. They gave me a retainer up front and had me bill them based on agreed-upon hourly amount. Ultimately the offer was based on that work. We ended up pressing pause as they were able to find help more local to them so I didn't join up but all around good experience. And it is certainly nice to be paid when it is work they may potentially use!
  - subhobroto 7y ago
    > they were able to find help more local to them so I didn't join up but all around good experience Dodged a bullet there. If there's a preference for a co-located candidate, that means their processes are not built to put a remote candidate on equal footing. Remote work is hard. NOTHING beats the bandwidth and fidelity of face to face conversations. As a result, remote workers need MORE help, not less to be successful. One way of MORE help is to have an async work flow. Let me guess: 1. this company has most of their employees live in the same city or state although they all "work from home most of the time"? 2. this company has a slack channel where their employees hang out and are almost always available?
    - 52-6F-62 7y ago
      In this case I think your assumptions were incorrect. They already had some remote staff and contacted me out of my expressed desire to work remotely. I was caught up in a critical project and needed more time however they needed the extra set of hands earlier than I could commit to. I'm guessing colocation was an added bonus because it is a bit easier. I think my largest concern was the sustainability of my rate for them in the long term being a small outfit. No regrets one way or the other. And like I said, they paid my retainer up front and it was definitely to my satisfaction.
      - subhobroto 7y ago
        > I'm guessing colocation was an added bonus because it is a bit easier. I would love to hear your thoughts on this: My opinion is that if there's a preference for a co-located candidate, that simply means their processes are not built to put a remote candidate on equal footing. Hence the preference. The fact that they have remote workers does not invalidate the fact that those remote workers are at a disadvantage. > I think my largest concern was the sustainability of my rate for them in the long term being a small outfit This is one more red flag: companies that have to work remotely because they have to not because they want to. There is a world of difference between a company that had to settle for remote work just because they could not afford to hire locally from a company that was built remote first because the founders believed in remote work. The former will always be on the lookout for local candidates, forever deprioritizing remote workers while the latter don't have any such discrimination in mind. I am glad it turned out well for you in the short term. Maybe you can follow up on how their remote workers are currently doing.

nostrebored 7y ago

I'm always shocked by the examples that come up when people talk about when talking about whiteboarding. DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications.

cblades 7y ago
That there is a plethora of implementations of and documentation available for. There is no need to memorize an implementation.
subhobroto 7y ago
Sure, DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications. I have coded DFS many times in my life, both at high school and atleast a dozen times each during my Bachelors and Masters and far more times at work because I do a lot of NLP and traversing a tree is a regular day at work. I can code DFS with you today, when out at a bar over a few beers. What I struggle with is to code DFS: 1. with people I don't know, am not comfortable with 2. who are looking at me expectantly, expecting me to fail at it (because that's what happens in most of the interviews they have been part of) 3. with time running out They only time I was able to perform DFS at an interview was at my first job out of college because just the month before I had prepared for my Data Structure final exam and could code DFS (or any of the HackerRank style algorithms) in my sleep no matter what even if all I had was a chisel and a slab of granite. I was not upset that I never had to actually code the DFS, invert a binary tree and find the minimum coin change (all the 3 questions I was asked for the job) ever after that - it was all C++/Python/Java CRUD, for 3 years because I had 0 professional experience with CRUD at that point so I would have never got the job in the first place. In fact the official policy at that place was to always use existing commercial libraries (and purchase them if necessary) because it was a CRUD shop and they had high turnover so any custom library would quickly become a liability.

mmazing 7y ago

I've interviewed several hundred people and gave them a take home test of a simple problem that we had encountered in our work. We asked them to write a simple API that would return the closest pharmacy, given a latitude and longitude supplied to the API, and gave them a CSV with 10 pharmacies. Language didn't matter, and we never even ran it to see if it actually worked. During the interview we would have them walk us through the code and explain what each part does and why they did whatever (this makes sure they actually wrote it and understand it). Then, we would throw curveballs like "what would you need to change in your code if there were 100,000 pharmacies", and "how would you need to change it to return the closest 10 pharmacies instead of the closest 1?" It's normally really difficult to get into someone's head about HOW they solve problems. I really felt like our method shined a light on their programming abilities. Also, how well they might anticipate future usage by building something smart to begin with and not code themselves into a corner.

subhobroto 7y ago

> Take home coding tests are fine. I really prefer them, honestly Home coding tests are great at gauging real life performance of a candidate, if done right, but in my limited exposure (I don't have a lot of time to work on coding tests just so I can collect data samples to back my opinion), coding tests are handled extremely poorly, with expectations being set extremely vaguely. Any realistic assignment requires a dialogue between the consumer and producer. Companies are trying to minimize the time/money they spend on each interview. Clarifications cost money. Hoping a company will engage in a "what really are your requirements? are you trying to test my basic coding competency or do you want to see the best I have got?" will elicit a very vague response, if any. If a company treats a home coding test as a real assignment and provides the candidate with the resources they need, I see no problem. Unfortunately from my personal experience and anecdotes from others, it's just not the case. > get rejected for not being able to cough up a DFS algorithm on the spot DFS is easy. BFS is easy. So is Coin changing, inverting binary trees and doing back flips. If you invest enough time and effort into it, you can do it. I have done DFS, BFS, Coin changing, inverting binary trees many times before: https://news.ycombinator.com/item?id=22368645 https://news.ycombinator.com/item?id=22368645 I am beginning to prefer DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews. I have to learn it once and can regurgitate until I forget them. I pay the heavy cost once upfront and amortize it. No such thing for a home coding test! I now have to pay the same cost over and over again, unable to amortize it because no two companies are solving the same problem unless you view it as a meta problem and in that case you might very well end up redesigning Django, RoR or a similar meta framework. Financially it would be cheaper to become a full time contractor and get paid to make POCs and demos unencumbered by NDAs. As a candidate a home coding test is extremely expensive compared to the traditional DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews.