6 ms·

Coding tests should be abolished. There's almost never any science or systematic approach to evaluation. How can you possibly evaluate a test without predefin

Coding tests should be abolished. There's almost never any science or systematic approach to evaluation. How can you possibly evaluate a test without predefined criteria for evaluating that makes the result good and what makes it bad? Whether or not the code is "good" is simply a matter of opinion in almost all cases. Another down side to coding tests is that if the assessor knows less than you about some area of coding and sees something they don't recognise or understand, then they might well negatively view what in fact is a much better way of doing things. And, much worse than anything else, you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all. And as for "Tic Tac Toe" as ANY sort of effective test topic, well, that's where you should pull out and find a different job to apply for, where they can at least formulate a relevant test topic and come up with tangible and defined criteria for measurement as well as a commitment from their end as to who will assess it, against what criteria and within what time frame and what feedback they will give you, if any. Ditch the Tic Tac Toe companies. I did two coding tests in recent years ... in one, their feedback was "they didn't like it", and the other one I worked on for 12 hours because I believe in doing a good job and meeting the requirements, and I heard nothing at all back - not a single word. Coding tests are, for the most part, bullshit.

coffee 6y ago
> you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all ...which is so very frustrating. It's painful to read things like this: > That was all we asked; we intentionally left it open-ended. Some people tried to impress us with huge, elaborate apps that used different services and engines. The people hiring viewed this as a trivial, small, meaningless ask. They are proud to have "intentionally left it open-ended." Some potential candidates did not (through their own example) view this as trivial, they went ahead and poured a ton of time into it and most likely were "replied with well wishes" as the company moved on. It's irresponsible.
- 9nGQluzmnq3M 6y ago
  Yes, companies need to be up front about roughly how much time they expect candidates to spend. That said, if a candidate spends 12 hours or 10,000 lines coding up tic-tac-toe, the company is not entirely to blame here...
  - coffee 6y ago
    > the company is not entirely to blame here... I clearly hear what you're saying, and I understand your stance on it. But I can't agree with it. The burden is on the folks hiring in my opinion. This is not a "real life" scenario. It should be a SHORT assesment (for both parties). If you were employed within a company as an engineer and you received a nebulus spec (or no spec at all), the burden is then on you to identify, educate, and communicate. Not during a pre-interview (for the chance and maybe being blessed enough to get a full interview with this world changing company).
    - 9nGQluzmnq3M 6y ago
      Your line of argumentation seems to argue the opposite: since it's the candidate who's the prospective engineer, wouldn't that imply that it's their job to clarify the nebulous spec? FWIW, many of the FAANGs are famous for asking intentionally underspecified questions, and the candidate preparing properly by figuring out what's in scope and what's not is part of the scoring. But at least in an interview you're timeboxed to 45 or 60 minutes!
      - throwaway298477 6y ago
        > wouldn't that imply that it's their job to clarify the nebulous spec? Maybe, except when you try to clarify and their answer is that it's open-ended -- and yet they want something super specific in the end. And the super specific things are just one of many tech opinions and not objectively good design. > FWIW, many of the FAANGs are famous for asking intentionally underspecified questions, and the candidate preparing properly by figuring out what's in scope and what's not is part of the scoring. Why is this a good thing though? Why is solving riddles in a professional interview somehow ok?
- mr_isomies 6y ago
  I can't agree more, having had to jump through these unpaid hoops myself for no carrots. What is the "price of a line of code" again? Between $40 and $100? Maybe the crucial parts of the "solution" should be masked and only revealed when that price is paid :)
- nerdponx 6y ago
  This, big time. Leaving people to guess at their own evaluation criteria is disrespectful, and implicitly biases in favor of people who have lots of free time/energy.
putsjoe 6y ago
I think an incredibly simple coding test is enough. At my previous role we were surprised just how many people looked good on paper for a development role but barely even attempted the first question on a 3 question HackerRank test.
jimmaswell 6y ago
I really can't think of anything more fair than a small coding test. You can demonstrate your ability on an actual problem instead of ridiculous whiteboard algorithm questions or having to have stuff on github.
dreamer7 6y ago
It depends on the objective of the coding test. If you are expecting a candidate to have a certain skill-set, having a simple timed coding test as the first round is the easiest way to filter out a lot of unqualified candidates. Stripe does this quite well for example. When I hire for my company, I do a live coding round with the candidate after an initial phone screen. Since I get far fewer applicants, this works out for me. The objective with the coding round is to understand more than their ability to write code in a particular language. It's a good way to learn how they communicate, how they approach a new problem, how they seek help, etc
- throwaway298477 6y ago
  Except when the candidate isn't told that they will have a timed coding interview in advance and in general what it would be. I've been asked to write a full-stack app from scratch, and I spent like half of the time debugging the webpack config, which could've easily been solved by having a template prepared in advance if I knew what I was doing. If anyone's reading these, please have the coders work off of an already existing template that you give them.
- coffee 6y ago
  > If you are expecting a candidate to have a certain skill-set, having a simple timed coding test as the first round is the easiest way to filter out a lot of unqualified candidates. Stripe does this quite well for example. The problem is 99% (or more) of the rand-o startups/companies that do this are not Stripe. They are not companies that people are fighting to become employed at. > this works out for me > their ability > how they communicate > how they approach > how they seek help What's ends up happening is you miss out on a lot of truly talented folks by putting the burden of proof 100% on them. > is the easiest way to filter out a lot of unqualified candidates It may be the easiest way for YOU. It's an important distinction as you (and this is not a personal attack, sorry if it sounds like it is) may not have the hiring skillset. It is a skill. Just like managing people, supporting software, being a CEO, etc...
kube-system 6y ago
It is very possible to have a systematic way to evaluate a coding test. I'm sure many people on this forum have been on the receiving end of such assignments while in school. But unlike school, companies usually have a disincentive to sharing their rubric and final evaluation.
m463 6y ago
I think writing a little code is reasonable as PART of a balanced interview process. As to the coding part, I have interviewed some people who seemed very together, but they could not write a simple program. I do mean simple, not some brain-cracking gauntlet of a test. Something you let them work on, in front of you, for maybe 10-15 minutes max.
- _vn5r 6y ago
  For example?
  - unishark 6y ago
    Here are things I have had CS grads demonstrate an inability to do within an hour or more timeframe: Make a loop that sums up numbers in an array. Write an algorithm that sorts a list of three numbers. I watched students spend an hour making a bunch of if statements to try and cover all cases... ultimately getting it wrong. Compute the accuracy by computing the number of matches between two lists and dividing between the length. Split a list into two lists. Generally the failure mode is to find something vaguely near the task, but overly-sophisticated from google. Then wrestle with trying to hack it into working by frantically scouring stackoverflow etc for their next code adjustment.
  - m463 6y ago
    sort a linked list. convert an ascii number to an integer.
    - throwaway298477 6y ago
      You're just describing leetcode here. Not something most people even do in their day-to-day. A lot of frontend devs don't even work with the DOM directly after having spent years with React. Could they do it? probably. Are they prepared for DOM-related manipulation questions on a whiteboard? nope.
      - unishark 6y ago
Traster 6y ago
My company runs a coding test, it's like "here are some interfaces we want a solution that detects Y and does Z". We don't compile or run the code that people submit. We're literally looking for if they ask the right questions, structure their design in anything like a sane way, and understand the fundamental restrictions of the technology we use. Oh and that previous paragraph? That's how I explain our coding test, I'm very clear what our standards are. >How can you possibly evaluate a test without predefined criteria for evaluating that makes the result good and what makes it bad? If it's even a slightly reasonable solution it's a pass. It doesn't need to be "good" it just needs to not be terrible. >Another down side to coding tests is that if the assessor knows less than you about some area of coding and sees something they don't recognise or understand, then they might well negatively view what in fact is a much better way of doing things. A correct solution is acheivable in between 50-100 lines of code. If you do something I don't understand I can look it up. In fact, people have done things that I don't understand and looked up. >And, much worse than anything else, you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all. Every candidate has a phone interview first and every candidate gets a response to the coding test. It's unusual to complete the test without e-mail back and forth with one of us for clarification (which is part of the test). >And as for "Tic Tac Toe" as ANY sort of effective test topic, First you wanted something that didn't take 12 hours, now you're complaining the test is too simple. You don't know what the company does, so how do you know the skills you need for a tictactoe solver aren't relevant? >I did two coding tests in recent years ... in one, their feedback was "they didn't like it", and the other one I worked on for 12 hours because I believe in doing a good job and meeting the requirements, and I heard nothing at all back - not a single word. Companies being bad at coding tests doesn't mean coding tests are bad. Companies are liable to ghost you at any point in the interview, that's nothing to do with a coding test. Our interview process is on-site and normally takes 1/2 visits, it will involve probably 8 person-hours of our engineering time + management. So yes, we want to make sure that candidates are atleast vaguely able to solve practical problems before they come in.

coffee 6y ago

> you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all ...which is so very frustrating. It's painful to read things like this: > That was all we asked; we intentionally left it open-ended. Some people tried to impress us with huge, elaborate apps that used different services and engines. The people hiring viewed this as a trivial, small, meaningless ask. They are proud to have "intentionally left it open-ended." Some potential candidates did not (through their own example) view this as trivial, they went ahead and poured a ton of time into it and most likely were "replied with well wishes" as the company moved on. It's irresponsible.

9nGQluzmnq3M 6y ago
Yes, companies need to be up front about roughly how much time they expect candidates to spend. That said, if a candidate spends 12 hours or 10,000 lines coding up tic-tac-toe, the company is not entirely to blame here...
- coffee 6y ago
  > the company is not entirely to blame here... I clearly hear what you're saying, and I understand your stance on it. But I can't agree with it. The burden is on the folks hiring in my opinion. This is not a "real life" scenario. It should be a SHORT assesment (for both parties). If you were employed within a company as an engineer and you received a nebulus spec (or no spec at all), the burden is then on you to identify, educate, and communicate. Not during a pre-interview (for the chance and maybe being blessed enough to get a full interview with this world changing company).
  - 9nGQluzmnq3M 6y ago
    Your line of argumentation seems to argue the opposite: since it's the candidate who's the prospective engineer, wouldn't that imply that it's their job to clarify the nebulous spec? FWIW, many of the FAANGs are famous for asking intentionally underspecified questions, and the candidate preparing properly by figuring out what's in scope and what's not is part of the scoring. But at least in an interview you're timeboxed to 45 or 60 minutes!
    - throwaway298477 6y ago
      > wouldn't that imply that it's their job to clarify the nebulous spec? Maybe, except when you try to clarify and their answer is that it's open-ended -- and yet they want something super specific in the end. And the super specific things are just one of many tech opinions and not objectively good design. > FWIW, many of the FAANGs are famous for asking intentionally underspecified questions, and the candidate preparing properly by figuring out what's in scope and what's not is part of the scoring. Why is this a good thing though? Why is solving riddles in a professional interview somehow ok?
mr_isomies 6y ago
I can't agree more, having had to jump through these unpaid hoops myself for no carrots. What is the "price of a line of code" again? Between $40 and $100? Maybe the crucial parts of the "solution" should be masked and only revealed when that price is paid :)
nerdponx 6y ago
This, big time. Leaving people to guess at their own evaluation criteria is disrespectful, and implicitly biases in favor of people who have lots of free time/energy.

putsjoe 6y ago

I think an incredibly simple coding test is enough. At my previous role we were surprised just how many people looked good on paper for a development role but barely even attempted the first question on a 3 question HackerRank test.

jimmaswell 6y ago

I really can't think of anything more fair than a small coding test. You can demonstrate your ability on an actual problem instead of ridiculous whiteboard algorithm questions or having to have stuff on github.

dreamer7 6y ago

It depends on the objective of the coding test. If you are expecting a candidate to have a certain skill-set, having a simple timed coding test as the first round is the easiest way to filter out a lot of unqualified candidates. Stripe does this quite well for example. When I hire for my company, I do a live coding round with the candidate after an initial phone screen. Since I get far fewer applicants, this works out for me. The objective with the coding round is to understand more than their ability to write code in a particular language. It's a good way to learn how they communicate, how they approach a new problem, how they seek help, etc

throwaway298477 6y ago
Except when the candidate isn't told that they will have a timed coding interview in advance and in general what it would be. I've been asked to write a full-stack app from scratch, and I spent like half of the time debugging the webpack config, which could've easily been solved by having a template prepared in advance if I knew what I was doing. If anyone's reading these, please have the coders work off of an already existing template that you give them.
coffee 6y ago
> If you are expecting a candidate to have a certain skill-set, having a simple timed coding test as the first round is the easiest way to filter out a lot of unqualified candidates. Stripe does this quite well for example. The problem is 99% (or more) of the rand-o startups/companies that do this are not Stripe. They are not companies that people are fighting to become employed at. > this works out for me > their ability > how they communicate > how they approach > how they seek help What's ends up happening is you miss out on a lot of truly talented folks by putting the burden of proof 100% on them. > is the easiest way to filter out a lot of unqualified candidates It may be the easiest way for YOU. It's an important distinction as you (and this is not a personal attack, sorry if it sounds like it is) may not have the hiring skillset. It is a skill. Just like managing people, supporting software, being a CEO, etc...

kube-system 6y ago

It is very possible to have a systematic way to evaluate a coding test. I'm sure many people on this forum have been on the receiving end of such assignments while in school. But unlike school, companies usually have a disincentive to sharing their rubric and final evaluation.

m463 6y ago

I think writing a little code is reasonable as PART of a balanced interview process. As to the coding part, I have interviewed some people who seemed very together, but they could not write a simple program. I do mean simple, not some brain-cracking gauntlet of a test. Something you let them work on, in front of you, for maybe 10-15 minutes max.

_vn5r 6y ago
For example?
- unishark 6y ago
  Here are things I have had CS grads demonstrate an inability to do within an hour or more timeframe: Make a loop that sums up numbers in an array. Write an algorithm that sorts a list of three numbers. I watched students spend an hour making a bunch of if statements to try and cover all cases... ultimately getting it wrong. Compute the accuracy by computing the number of matches between two lists and dividing between the length. Split a list into two lists. Generally the failure mode is to find something vaguely near the task, but overly-sophisticated from google. Then wrestle with trying to hack it into working by frantically scouring stackoverflow etc for their next code adjustment.
- m463 6y ago
  sort a linked list. convert an ascii number to an integer.
  - throwaway298477 6y ago
    You're just describing leetcode here. Not something most people even do in their day-to-day. A lot of frontend devs don't even work with the DOM directly after having spent years with React. Could they do it? probably. Are they prepared for DOM-related manipulation questions on a whiteboard? nope.
    - unishark 6y ago

Traster 6y ago

My company runs a coding test, it's like "here are some interfaces we want a solution that detects Y and does Z". We don't compile or run the code that people submit. We're literally looking for if they ask the right questions, structure their design in anything like a sane way, and understand the fundamental restrictions of the technology we use. Oh and that previous paragraph? That's how I explain our coding test, I'm very clear what our standards are. >How can you possibly evaluate a test without predefined criteria for evaluating that makes the result good and what makes it bad? If it's even a slightly reasonable solution it's a pass. It doesn't need to be "good" it just needs to not be terrible. >Another down side to coding tests is that if the assessor knows less than you about some area of coding and sees something they don't recognise or understand, then they might well negatively view what in fact is a much better way of doing things. A correct solution is acheivable in between 50-100 lines of code. If you do something I don't understand I can look it up. In fact, people have done things that I don't understand and looked up. >And, much worse than anything else, you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all. Every candidate has a phone interview first and every candidate gets a response to the coding test. It's unusual to complete the test without e-mail back and forth with one of us for clarification (which is part of the test). >And as for "Tic Tac Toe" as ANY sort of effective test topic, First you wanted something that didn't take 12 hours, now you're complaining the test is too simple. You don't know what the company does, so how do you know the skills you need for a tictactoe solver aren't relevant? >I did two coding tests in recent years ... in one, their feedback was "they didn't like it", and the other one I worked on for 12 hours because I believe in doing a good job and meeting the requirements, and I heard nothing at all back - not a single word. Companies being bad at coding tests doesn't mean coding tests are bad. Companies are liable to ghost you at any point in the interview, that's nothing to do with a coding test. Our interview process is on-site and normally takes 1/2 visits, it will involve probably 8 person-hours of our engineering time + management. So yes, we want to make sure that candidates are atleast vaguely able to solve practical problems before they come in.