5 ms·

Take home coding tests as a 0th round interview step can be extremely effective if they are short (30m-1h), relatively easy pulse checks (a step or two above Fi

Take home coding tests as a 0th round interview step can be extremely effective if they are short (30m-1h), relatively easy pulse checks (a step or two above FizzBuzz level). Any more involved than that and they are insulting to candidates, thus driving away talent. I worked for a company that administered a simple half-hour take home coding challenge before any interviews. The problem was simple but not trivial — print Fibonacci numbers, take a string of brackets and print if they were properly nested (e.g. '()(())' is OK, '(()(' is not). We had essentially zero attrition of candidates with good CVs who refused to take the test. On the other hand, we disqualified >80% of applicants from the test alone, saving an enormous amount of interviewer time. It would have sucked to have phone screened all those candidates and spent half an hour of company time on each one to discover they can't code at all.

yellowcake0 4y ago
It sounds like it is working for that company so more power to them I suppose, and I've certainly done my fair share of coding tests and take home assignments, however I consider it personally disrespectful to automate the initial interview round in this way. If you're asking me to put in work, even if it's an ostensibly short task, I want to talk to someone first.
- marcosdumay 4y ago
  Well, ok, everybody sees things their own way. Personally, I would be glad to do a short take home if it's clear that it will reduce the bullshit analysis of curriculum. I'd be even happier if it replaced a whiteboard FizzBuzz, and would raise good flags if it was a mathy problem like the ones the OP talked about, instead of unmotivated terminal or memory juggling.
- kube-system 4y ago
  I agree, it's disrespectful. I always give people the opportunity to speak with a human about the job before requiring they invest any time on an assignment. Seeing if someone has the skills to do the job is secondary to determining if the job is a fit in plain-english conversation.
bell-cot 4y ago
> ...print Fibonacci numbers, take a string of brackets and print if they were properly nested... I'm nowhere near elite...but hopefully none of your candidates need anywhere near 1/2 hour to code stuff that simple.
- monksy 4y ago
  Add testing, doc, application backend, etc. You'll go beyond 0.5hrs.
  - berkle4455 4y ago
    And if a candidate submitted that for an algorithm screener (which at my company we occasionally see similar), they get rejected.
    - Thrymr 4y ago
      I would hope that your instructions are very explicit on what is expected if you are rejecting candidates for submitting _too complete a solution_ to the problem you gave them.
  - lisasays 4y ago
    "Application backend" for Fibanocci?
    - monksy 4y ago
      I've never seen a take home assignment that asks for a fib. I've only have been asked about fib in whiteboarding sessions. Usually it's a "Please write a program to solve this problem with the following rules" or "Please write a service that services x,y,x"
- danwee 4y ago
  It depends. The snippet of code itself, yeah, it could take less than 30min. But then what, do you submit just as it is? No, you run "main.py/main.go/main.whatever". Do we need command line parameters? Maybe? Better to set them by default? Ah, well, then we probably need a "-help" so that the parameters are self documented... or do we also add them to the README.md? Because we are adding a README.md, right? And, do we setup the git repo (because we are submitting a git repo and not a zip file, right?) the "right way"? That means different things with different programming languages. Ah, what about the tests? Should we add an easy way to run the tests (e.g., Makefile?) or do we rely on the default way to run tests in the programming language we have chosen? But what if we choose a programming language the company I'm applying for doesn't use? Well, better to document that, right? Wait, do we need to print the results in the standard output just like that? The specification of the problem doesn't say much so I better e-mail them to ask for further instructions (that will show them that I don't assume anything and I always avoid doing the wrong thing! That probably gives me points at the end of the day). 1h has passed since I sent the email and no response from them so far... umm, should I just do it my way? Ummm And so on...
i_love_cookies 4y ago
[dead]
monksy 4y ago
This is one of those things that you'd be amazed once you start looking at interviewer's ability to judge effort. Often times the questions asked or expected of is just excessive. They'll ask problems that if you didn't know the trick or (years of ) research behind it, you'd never solve it. They'll give a take home asking for a full service and full deployment on K8s and claim "it should only take 6 hours".
- ozim 4y ago
  I fail to see the problem. If companies look for people who have years of experience in k8s or for people who don’t trip on something specific it is fair game. If they look for full stack dev and then grill on some obscure details of one tech stack piece it is also their problem.
  - yuliyp 4y ago
    Someone can have lots of k8s experience but not have needed to set things up end-to-end. Most people generally are familiar with "X as used at my job" which is very different from "X as what happens when I use it from scratch". This is true not just for orchestration systems like k8s, but also programming languages: people are used to the libraries they have for solving common problems, which are not necessarily OSS / standard libraries.
    - ozim 4y ago
      We discuss "experience with" which for me is really vague. Is company looking for experience like "someone who could connect to k8s and debug application"? Is company looking for "someone who can setup k8s and have networking up and running"? How do deal with people claiming "experience with" where someone set something up for them and they were using stuff but never had to dig into details. In the end it is also like driving - while driving is driving then "F1 driving" is so much more and while F1 driver is not an engineer you still expect him to understand how ICE engine works and bunch of other technical details. For everyday driver you expect no more than "put in fuel, turn the key, throttle and you drive, break you stop".
  - 4y ago
ladberg 4y ago
I can echo this and also push back against most of the other anti-test comments in the thread. I agree that an unbounded and abstract take home test is bad, but we use an online test provider (leetcode-esque) with some custom relatively easy questions. We only do it for certain demographics (i.e. interns and fresh grads) while experienced candidates skip it, and it certainly saves a TON of interviewer time. Even for the internship alone we get tens of thousands of applications and it's pretty intractable to review all of their resumes manually in a relatively small window of time, so there needs to be some type of automated system to discard a majority of candidates.
gwright 4y ago
FWIW, I've had good success with these two approaches: * code review: Screen share with the candidate a small code snippet in a language/framework that they are comfortable with that needs improvement. Have candidate talk through what feedback they would provide in the context of a code review to improve the code. * code pairing: Pair candidate with an engineer on the team to improve some existing code (add a simple feature). Both of these approaches have been very helpful at understanding how skilled a candidate is, how familiar they are with a language or framework, how they provide constructive feedback, etc. Code snippets can be designed to encourage discussion of security practices (SQL injection, for example), capacity planning (5 transactions/day, /hour, /sec?), algorithmic complexity, etc.
vineyardmike 4y ago
I was recently looking for a job and a company I interviewed at had the first round as a take home test. You had to write a CLI program that called their stupid-simple rest API (takes a number, returns 1+ numbers streamed) in the language/runtime the company uses. They said it shouldn’t take over 2-3 hrs. I set a 2Hr timer, completed it and got partway through tests, and stopped (leaving unfinished tests or missing tests etc). I spent 30 min writing todo comments (“TODO: refactor this method for easier test-ability”) and documentation. 2.5hr total, then moved on to next in-person round. Their in person interview questions are all based on either my code or their server side code for this test project. It was great because I already knew about it from the take home. They explicitly asked for a simple, working cli program in the language they use, in a reasonable time box. They said if you have time, consider test-ability, consider performance (the service streamed numbers so you needed to not store them in-memory), and other things a “real” program would need to consider. They gave themselves amble signal for a simple project, and set the time box low enough that most anyone can find that time. I will absolutely be using a similar process for hiring at my current company.
tengbretson 4y ago
An hour to implement a one-liner? const test = toTest => { try { new Function(`return [ ${toTest.replaceAll('(', '[').replaceAll(')', '],')} ]`)(); return true } catch (e) { return false; } }
- lisasays 4y ago
  Seems like a horrible design choice to implement something so error-prone (and not immediately, visually obvious) as a one-liner -- and involving both dynamic code generation, -and- the awkward and performance-killing use of exception handling as a loop control, no less.

yellowcake0 4y ago

It sounds like it is working for that company so more power to them I suppose, and I've certainly done my fair share of coding tests and take home assignments, however I consider it personally disrespectful to automate the initial interview round in this way. If you're asking me to put in work, even if it's an ostensibly short task, I want to talk to someone first.

marcosdumay 4y ago
Well, ok, everybody sees things their own way. Personally, I would be glad to do a short take home if it's clear that it will reduce the bullshit analysis of curriculum. I'd be even happier if it replaced a whiteboard FizzBuzz, and would raise good flags if it was a mathy problem like the ones the OP talked about, instead of unmotivated terminal or memory juggling.
kube-system 4y ago
I agree, it's disrespectful. I always give people the opportunity to speak with a human about the job before requiring they invest any time on an assignment. Seeing if someone has the skills to do the job is secondary to determining if the job is a fit in plain-english conversation.

bell-cot 4y ago

> ...print Fibonacci numbers, take a string of brackets and print if they were properly nested... I'm nowhere near elite...but hopefully none of your candidates need anywhere near 1/2 hour to code stuff that simple.

monksy 4y ago
Add testing, doc, application backend, etc. You'll go beyond 0.5hrs.
- berkle4455 4y ago
  And if a candidate submitted that for an algorithm screener (which at my company we occasionally see similar), they get rejected.
  - Thrymr 4y ago
    I would hope that your instructions are very explicit on what is expected if you are rejecting candidates for submitting _too complete a solution_ to the problem you gave them.
- lisasays 4y ago
  "Application backend" for Fibanocci?
  - monksy 4y ago
    I've never seen a take home assignment that asks for a fib. I've only have been asked about fib in whiteboarding sessions. Usually it's a "Please write a program to solve this problem with the following rules" or "Please write a service that services x,y,x"
danwee 4y ago
It depends. The snippet of code itself, yeah, it could take less than 30min. But then what, do you submit just as it is? No, you run "main.py/main.go/main.whatever". Do we need command line parameters? Maybe? Better to set them by default? Ah, well, then we probably need a "-help" so that the parameters are self documented... or do we also add them to the README.md? Because we are adding a README.md, right? And, do we setup the git repo (because we are submitting a git repo and not a zip file, right?) the "right way"? That means different things with different programming languages. Ah, what about the tests? Should we add an easy way to run the tests (e.g., Makefile?) or do we rely on the default way to run tests in the programming language we have chosen? But what if we choose a programming language the company I'm applying for doesn't use? Well, better to document that, right? Wait, do we need to print the results in the standard output just like that? The specification of the problem doesn't say much so I better e-mail them to ask for further instructions (that will show them that I don't assume anything and I always avoid doing the wrong thing! That probably gives me points at the end of the day). 1h has passed since I sent the email and no response from them so far... umm, should I just do it my way? Ummm And so on...

i_love_cookies 4y ago

[dead]

monksy 4y ago

This is one of those things that you'd be amazed once you start looking at interviewer's ability to judge effort. Often times the questions asked or expected of is just excessive. They'll ask problems that if you didn't know the trick or (years of ) research behind it, you'd never solve it. They'll give a take home asking for a full service and full deployment on K8s and claim "it should only take 6 hours".

ozim 4y ago
I fail to see the problem. If companies look for people who have years of experience in k8s or for people who don’t trip on something specific it is fair game. If they look for full stack dev and then grill on some obscure details of one tech stack piece it is also their problem.
- yuliyp 4y ago
  Someone can have lots of k8s experience but not have needed to set things up end-to-end. Most people generally are familiar with "X as used at my job" which is very different from "X as what happens when I use it from scratch". This is true not just for orchestration systems like k8s, but also programming languages: people are used to the libraries they have for solving common problems, which are not necessarily OSS / standard libraries.
  - ozim 4y ago
    We discuss "experience with" which for me is really vague. Is company looking for experience like "someone who could connect to k8s and debug application"? Is company looking for "someone who can setup k8s and have networking up and running"? How do deal with people claiming "experience with" where someone set something up for them and they were using stuff but never had to dig into details. In the end it is also like driving - while driving is driving then "F1 driving" is so much more and while F1 driver is not an engineer you still expect him to understand how ICE engine works and bunch of other technical details. For everyday driver you expect no more than "put in fuel, turn the key, throttle and you drive, break you stop".
- 4y ago

ladberg 4y ago

I can echo this and also push back against most of the other anti-test comments in the thread. I agree that an unbounded and abstract take home test is bad, but we use an online test provider (leetcode-esque) with some custom relatively easy questions. We only do it for certain demographics (i.e. interns and fresh grads) while experienced candidates skip it, and it certainly saves a TON of interviewer time. Even for the internship alone we get tens of thousands of applications and it's pretty intractable to review all of their resumes manually in a relatively small window of time, so there needs to be some type of automated system to discard a majority of candidates.

gwright 4y ago

FWIW, I've had good success with these two approaches: * code review: Screen share with the candidate a small code snippet in a language/framework that they are comfortable with that needs improvement. Have candidate talk through what feedback they would provide in the context of a code review to improve the code. * code pairing: Pair candidate with an engineer on the team to improve some existing code (add a simple feature). Both of these approaches have been very helpful at understanding how skilled a candidate is, how familiar they are with a language or framework, how they provide constructive feedback, etc. Code snippets can be designed to encourage discussion of security practices (SQL injection, for example), capacity planning (5 transactions/day, /hour, /sec?), algorithmic complexity, etc.

vineyardmike 4y ago

I was recently looking for a job and a company I interviewed at had the first round as a take home test. You had to write a CLI program that called their stupid-simple rest API (takes a number, returns 1+ numbers streamed) in the language/runtime the company uses. They said it shouldn’t take over 2-3 hrs. I set a 2Hr timer, completed it and got partway through tests, and stopped (leaving unfinished tests or missing tests etc). I spent 30 min writing todo comments (“TODO: refactor this method for easier test-ability”) and documentation. 2.5hr total, then moved on to next in-person round. Their in person interview questions are all based on either my code or their server side code for this test project. It was great because I already knew about it from the take home. They explicitly asked for a simple, working cli program in the language they use, in a reasonable time box. They said if you have time, consider test-ability, consider performance (the service streamed numbers so you needed to not store them in-memory), and other things a “real” program would need to consider. They gave themselves amble signal for a simple project, and set the time box low enough that most anyone can find that time. I will absolutely be using a similar process for hiring at my current company.

tengbretson 4y ago

An hour to implement a one-liner? const test = toTest => { try { new Function(`return [ ${toTest.replaceAll('(', '[').replaceAll(')', '],')} ]`)(); return true } catch (e) { return false; } }

lisasays 4y ago
Seems like a horrible design choice to implement something so error-prone (and not immediately, visually obvious) as a one-liner -- and involving both dynamic code generation, -and- the awkward and performance-killing use of exception handling as a loop control, no less.