3 ms·

> Well, empirically we're getting quite a lot of people attempting it! > Without something like this we'd just have 100s-1000s of unstructured resumes in diffe

> Well, empirically we're getting quite a lot of people attempting it! > Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. So instead of hundreds or thousands of unstructured resumes in different formats, you have hundreds or thousands of completed programming tests. Even assuming that you automatically filter out submissions that fail testing, given that there are a lot of ways to solve a problem, you'd still be stuck evaluating potentially hundreds of correct submissions. More importantly, a test doesn't seem to be aligned with your top priority: > ...but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success. Are you assuming that all of the folks most accomplished in the real world are going to complete your programming test successfully? Since you ultimately seem to be interested in folks who can build dashboards, I would think you'd be far better off asking candidates to build a dashboard prototype using sample data you provide. There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.

dlss 13y ago
> You'd still be stuck evaluating potentially hundreds of correct submissions. I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy. > There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application. Agree! I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff. (ie https://web.archive.org/web/20130114163457/https://raganwald.posterous.com/i-dont-hire-unlucky-people https://web.archive.org/web/20130114163457/https://raganwald... meets http://en.wikipedia.org/wiki/Goodhart's_law http://en.wikipedia.org/wiki/Goodhart's_law)
- lifeisstillgood 13y ago
  To be honest I think it's the other way round - if you can do programming tests all day long you will succeed at most coding jobs. However there are a lot of developers who can't tackle an interview that with a nights sleep and no pressure can easily code a intuitive and effective architecture. Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
  - dlss 13y ago
    I'm not sure what kind of coding work you do, but I assure you the vast majority of web development work has little resemblance to coding questions. You are almost never designing an algorithm: that was already done for you by whoever designed the library you are using. Majority of the tasks IME: - code reading & refactoring - figuring out why two libraries aren't playing nice - bug triangulation - optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind) - repetitive/simple/obvious bits of jQuery Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).
    - joncooper 13y ago
      Another key: - Communicating clearly and calmly with other people
      - dlss 13y ago
        You're right, we need to add social skills and domain knowledge to the list.
    - lifeisstillgood 13y ago
      That's terribly negative. If you are doing agency work maybe (same website, same functionality, just different skin) but that vast vast open plains of http-based development are around and going to be around joining parts of the business, public world together in ways they do not know or imagine. Yes, binary trees are pretty much coded for you, and you would be a fool to roll your own hashing algorithm for production, but thats still not a reason to keep your head down. The next decade is going to be fun. Trust me

dlss 13y ago

> You'd still be stuck evaluating potentially hundreds of correct submissions. I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy. > There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application. Agree! I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff. (ie https://web.archive.org/web/20130114163457/https://raganwald.posterous.com/i-dont-hire-unlucky-people https://web.archive.org/web/20130114163457/https://raganwald... meets http://en.wikipedia.org/wiki/Goodhart's_law http://en.wikipedia.org/wiki/Goodhart's_law)

lifeisstillgood 13y ago
To be honest I think it's the other way round - if you can do programming tests all day long you will succeed at most coding jobs. However there are a lot of developers who can't tackle an interview that with a nights sleep and no pressure can easily code a intuitive and effective architecture. Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
- dlss 13y ago
  I'm not sure what kind of coding work you do, but I assure you the vast majority of web development work has little resemblance to coding questions. You are almost never designing an algorithm: that was already done for you by whoever designed the library you are using. Majority of the tasks IME: - code reading & refactoring - figuring out why two libraries aren't playing nice - bug triangulation - optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind) - repetitive/simple/obvious bits of jQuery Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).
  - joncooper 13y ago
    Another key: - Communicating clearly and calmly with other people
    - dlss 13y ago
      You're right, we need to add social skills and domain knowledge to the list.
  - lifeisstillgood 13y ago
    That's terribly negative. If you are doing agency work maybe (same website, same functionality, just different skin) but that vast vast open plains of http-based development are around and going to be around joining parts of the business, public world together in ways they do not know or imagine. Yes, binary trees are pretty much coded for you, and you would be a fool to roll your own hashing algorithm for production, but thats still not a reason to keep your head down. The next decade is going to be fun. Trust me