4 ms·

I feel like coding challenges like these are contrived and really prove nothing. Cool, you can create a "palindrome index." Can you effectively lazy load assets

I feel like coding challenges like these are contrived and really prove nothing. Cool, you can create a "palindrome index." Can you effectively lazy load assets in an app on demand? Can you architect your CSS in a way that only the relevant styles show up on a page? Do you know why a standard for loop is faster in most cases than a for in loop? Tests like what this site is trying to do don't really prove anything other than you're good at solving puzzles with code whereas as an employer I'm more interested in the quality of your applicable code and architecting thought processes. I'm not saying these tests are totally useless. Obviously it demonstrates a dev's ability to code within one or more languages, but they're hardly enough to make a hire off of. When it comes to building actual product, these tests always fall short.

opendais 12y ago
If your core business is built around a specific CS problem [e.g. Uber would be the traveling salesman, Google is searching with NLP], picking candidates that demonstrate they can solve that sort of problem effectively [not perfectly, just 'good enough'] is pretty relevant. [e.g. Code Combat's Greed tournament would be a good recruiting platform for finding developers for Uber that worked on the backend since the traveling salesman problem seems really relevant to them] However, I think you are right in that most of them are useless because they don't focus on that sort of detail. EDIT: Okay, I had a bad example. Sorry.
- Punoxysm 12y ago
  I guarantee the amount of time spent on traveling salesman problems at Uber is a miniscule fraction of all development time.
  - opendais 12y ago
    Oh, I'm sure. But it would be relevant which was my point. Someone at Uber at some point had to solve that sort of problem. Many of the questions I've been asked in interviews are more or less, "Someone had to do X at some point as part of our product, how would you do it?" The goal is to figure out if you are competent at solving business problems logically that are relevant to the business in some way.
    - potatolicious 12y ago
      Would it be though? Amazon spends a lot of time on the traveling salesman problem too, but only the extreme high end architect level people, on a very tiny number of teams. 99.9% of engineers at Amazon are writing, well, downright normal code. It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce. It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position.
      - opendais 12y ago
        Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me. It doesn't to other people. Think of it as a more relevant FizzBuzz. > It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce. > It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position. Is a naive solution to TSP really that hard?
        potatolicious 12y ago
        > "Is a naive solution to TSP really that hard?" Why do we want a naive solution to TSP? The naive solution to TSP has no applicability to anything - it's purely a thought exercise in the same vein as "why are manhole covers round" and "how many jelly beans are in this jar". Remember that 10 years ago we thought those questions were "predictors" of programmer ability. Turns out that was full of crap. So your notion of testing someone's suitability for a job is to ask them to solve a problem unlike any they will encounter in the job, expecting them to generate a solution that would be wildly insufficient even if the problem was relevant to their job? In the mean time I rarely see companies testing for abilities that are used on a regular basis in these jobs: the ability to architect, knowledge and familiarity with best practices and design patterns, writing testable code, etc etc. All of these skills are far from universal, but yet we spend no time ensuring they're there. No, we blow the valuable 45-60 minutes we have with candidates twiddling around with TSP. We ask candidates simplified things from what goes in IRL, because we don't have all day. That can't be helped - the least we can do is make sure what we ask is actually a simplified version of what the candidate will be responsible for, instead of simplified versions of things the candidate will never touch, ever. > "Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me." It doesn't to me. More accurately, it doesn't seem relevant to the majority of coding jobs out there. In reality the number of jobs that ever involve implementing real algorithms is really low. Even in research-heavy companies like Google it's limited to a small subset of employees in a small subset of teams. The vast majority of everyone will not write a substantial algorithm from scratch in their day job. Ever. So why are we testing for this ability?
- spamizbad 12y ago
  Uber isn't built around traveling salesman. When you put in your location and ask Uber for a ride it effectively broadcasts your intent to nearby drivers (with some filtering logic put in place based on your rating and the driver's rating). AFIAK the app doesn't even take into account 1-way streets when computing anything.
nilkn 12y ago
The idea has always been to judge ability and potential rather than domain-specific knowledge. The problem is that, as with most tests, these can be gamed. Every programmer under the sun now knows about FizzBuzz. So now there's an entire art dedicated to the craft of inventing new programming interview questions. Inevitably, the questions which were supposed to be the antithesis to trivia start to become their own trivia. Really, though, it's just a hard problem. There is no easy way to really accurately assess a programmer you've never met or worked with. Your specific questions could easily turn down an incredibly talented programmer who just hasn't done much front-end work before but could pick it up in a few afternoons.
anilgulecha 12y ago
Anil from HRX here. Without getting into all the details this early: we're building tools to test for exactly the things you've stated. There are tasks/competencies that can only be tested via manual intervention. But things that can be automated are in our roadmap.
vellum 12y ago
I'm not saying these tests are totally useless. They're also easy to fake. If this became widely adopted, you could hire someone on Fiverr to boost your score.
- anilgulecha 12y ago
  Sure, anything popular can be gamed. And countermeasures will in-turn be applied (checking for plagiarism, etc). The point to note is that there is (usually) a job on the line, and a programming challenge is not the only part of the screening process -- it will usually be followed by a live coding session/interview. So the question is would someone risk gaming the process, when the chance of getting caught is high. We think not.
  - ultimatedelman 12y ago
    ...but when a programmer who has gamed the system gets an interview and then is discovered to be a fraud, doesn't that defeat the purpose of using this service in the first place (to save thousands of interview hours, per the article)?
    - anilgulecha 12y ago
      Some locks are broken and burglars get in. But that doesn't mean the business or purpose of locks is defeated.

opendais 12y ago

If your core business is built around a specific CS problem [e.g. Uber would be the traveling salesman, Google is searching with NLP], picking candidates that demonstrate they can solve that sort of problem effectively [not perfectly, just 'good enough'] is pretty relevant. [e.g. Code Combat's Greed tournament would be a good recruiting platform for finding developers for Uber that worked on the backend since the traveling salesman problem seems really relevant to them] However, I think you are right in that most of them are useless because they don't focus on that sort of detail. EDIT: Okay, I had a bad example. Sorry.

Punoxysm 12y ago
I guarantee the amount of time spent on traveling salesman problems at Uber is a miniscule fraction of all development time.
- opendais 12y ago
  Oh, I'm sure. But it would be relevant which was my point. Someone at Uber at some point had to solve that sort of problem. Many of the questions I've been asked in interviews are more or less, "Someone had to do X at some point as part of our product, how would you do it?" The goal is to figure out if you are competent at solving business problems logically that are relevant to the business in some way.
  - potatolicious 12y ago
    Would it be though? Amazon spends a lot of time on the traveling salesman problem too, but only the extreme high end architect level people, on a very tiny number of teams. 99.9% of engineers at Amazon are writing, well, downright normal code. It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce. It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position.
    - opendais 12y ago
      Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me. It doesn't to other people. Think of it as a more relevant FizzBuzz. > It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce. > It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position. Is a naive solution to TSP really that hard?
      - potatolicious 12y ago
        > "Is a naive solution to TSP really that hard?" Why do we want a naive solution to TSP? The naive solution to TSP has no applicability to anything - it's purely a thought exercise in the same vein as "why are manhole covers round" and "how many jelly beans are in this jar". Remember that 10 years ago we thought those questions were "predictors" of programmer ability. Turns out that was full of crap. So your notion of testing someone's suitability for a job is to ask them to solve a problem unlike any they will encounter in the job, expecting them to generate a solution that would be wildly insufficient even if the problem was relevant to their job? In the mean time I rarely see companies testing for abilities that are used on a regular basis in these jobs: the ability to architect, knowledge and familiarity with best practices and design patterns, writing testable code, etc etc. All of these skills are far from universal, but yet we spend no time ensuring they're there. No, we blow the valuable 45-60 minutes we have with candidates twiddling around with TSP. We ask candidates simplified things from what goes in IRL, because we don't have all day. That can't be helped - the least we can do is make sure what we ask is actually a simplified version of what the candidate will be responsible for, instead of simplified versions of things the candidate will never touch, ever. > "Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me." It doesn't to me. More accurately, it doesn't seem relevant to the majority of coding jobs out there. In reality the number of jobs that ever involve implementing real algorithms is really low. Even in research-heavy companies like Google it's limited to a small subset of employees in a small subset of teams. The vast majority of everyone will not write a substantial algorithm from scratch in their day job. Ever. So why are we testing for this ability?
spamizbad 12y ago
Uber isn't built around traveling salesman. When you put in your location and ask Uber for a ride it effectively broadcasts your intent to nearby drivers (with some filtering logic put in place based on your rating and the driver's rating). AFIAK the app doesn't even take into account 1-way streets when computing anything.

nilkn 12y ago

The idea has always been to judge ability and potential rather than domain-specific knowledge. The problem is that, as with most tests, these can be gamed. Every programmer under the sun now knows about FizzBuzz. So now there's an entire art dedicated to the craft of inventing new programming interview questions. Inevitably, the questions which were supposed to be the antithesis to trivia start to become their own trivia. Really, though, it's just a hard problem. There is no easy way to really accurately assess a programmer you've never met or worked with. Your specific questions could easily turn down an incredibly talented programmer who just hasn't done much front-end work before but could pick it up in a few afternoons.

anilgulecha 12y ago

Anil from HRX here. Without getting into all the details this early: we're building tools to test for exactly the things you've stated. There are tasks/competencies that can only be tested via manual intervention. But things that can be automated are in our roadmap.

vellum 12y ago

I'm not saying these tests are totally useless. They're also easy to fake. If this became widely adopted, you could hire someone on Fiverr to boost your score.

anilgulecha 12y ago
Sure, anything popular can be gamed. And countermeasures will in-turn be applied (checking for plagiarism, etc). The point to note is that there is (usually) a job on the line, and a programming challenge is not the only part of the screening process -- it will usually be followed by a live coding session/interview. So the question is would someone risk gaming the process, when the chance of getting caught is high. We think not.
- ultimatedelman 12y ago
  ...but when a programmer who has gamed the system gets an interview and then is discovered to be a fraud, doesn't that defeat the purpose of using this service in the first place (to save thousands of interview hours, per the article)?
  - anilgulecha 12y ago
    Some locks are broken and burglars get in. But that doesn't mean the business or purpose of locks is defeated.