3 ms·

I’m 75% sure this is a joke. If it’s not, this is the most ridiculous interview process I’ve ever heard of. Despite what the author says, this type of intervie

I’m 75% sure this is a joke. If it’s not, this is the most ridiculous interview process I’ve ever heard of. Despite what the author says, this type of interview is definitely something you can train for. If it’s real, and successful the author has essentially created a test for people who have spent a lot of time in competitive programming, or who spent a lot of time on interview prep.

qPM9l3XJrF 5y ago
Does the author claim this isn't something you can train for? I've given hundreds of coding interviews. I always timed the candidate (with their knowledge). So I have a general idea of the distribution of programming speed, and my guess for this particular problem is to solve it in 10 minutes, you need both talent and practice... just one isn't enough. So asking people to solve it in 10 minutes ends up filtering out untalented people who have practiced a lot (and talented people who haven't practiced as much)
- void_mint 5y ago
  I'd much rather my engineers have spent their time practicing their actual jobs and not brainteaser interviews. To wit, the interview questions I ask are usually in line with the role they're assuming.
  - qPM9l3XJrF 5y ago
    The author wrote this post in the context of hiring at a "hard technology startup". I interpret that as a place where engineers are expected to be able to do cutting edge computer science research (it's not a "well-established [role] to do specialized tasks"). It's not clear to me what sort of questions you would ask "in line with the role they're assuming" for this scenario. One idea here is to give candidates an open problem in CS and ask them to solve it. Which is a nice idea, but one of the other things I learned through administering hundreds of interviews is that open-ended questions aren't great interview problems because they depend a lot on creativity/lateral thinking/insight, which highly benefits from being in a relaxed frame of mind. So these questions end up being a test of how relaxed the candidate is. Stress creates tunnel vision, which is terrible for generating interesting research ideas but it's OK for solving these kind of leetcode problems. So checking for a very high level of programming aptitude, as a proxy for CS research ability, is an approach which is a bit more fair to candidates who are stressed out by interviews. (I also think it is a pretty decent proxy, because doing great CS research requires you to quickly & fluidly generate & evaluate algorithms / data structures which might solve your problem, which is a big part of what a great leetcoder does.) If you're targeting a very high level of generic programming aptitude, it is arguably most fair to make use of a standard method of measuring it. Leetcode problems are the industry standard for measuring programming aptitude. People know to practice them a lot and they know what to expect. If you came up with your own unique way to measure programming aptitude, that would create an even greater burden on candidates to practice (they'd have to do a whole different sort of practice in order to succeed in your interview), and also create anxiety due to an unexpected interview format. I think a lot of readers are overreacting because they didn't pay enough attention to this bit: "It's applicable if you're building an extraordinary team at a hard technology startup." The vast majority of companies in SV are not doing hard technology and don't need an extraordinary team. People should not feel inadequate if they aren't capable of improving the state of the art in technical areas of CS such as databases. This is ordinarily the domain of PhDs, and filtering for demonstrated algorithmic aptitude (as opposed to academic credentials) is actually a pretty egalitarian approach.
    - void_mint 5y ago
      So let's be clear. You're honing in on this block: > "It's applicable if you're building an extraordinary team at a hard technology startup." And you're accepting that a timed question about tic tac toe is enough to prove you're capable of being on an "extraordinary team" at a "hard technology startup"? Really?
      - qPM9l3XJrF 5y ago
        Yes, I think to a large degree being able to build hard technology is a matter of being extremely fluent with writing code and reasoning about data structures and algorithms, as I stated. Being able to solve this problem in 10 minutes seems like a hard-to-fake demonstration of such fluency. When I think about CS open problems and when I solve leetcode problems with algorithmic content to them like this one, it feels like I'm using the same part of my brain (or at least there is significant overlap). If you don't agree with me that the fluency I described is a significant asset to advancing the state of the art in CS, what do you think a significant asset is?
        void_mint 5y ago
        > Yes, I think to a large degree being able to build hard technology is a matter of being extremely fluent with writing code and reasoning about data structures and algorithms Sure. Again. I don't really think an algorithm that shows up as an introduction to algorithm proves much more than a person read "Intro to Algorithms". So again, a timed introductory problem proves some elite technical skill? > If you don't agree with me that the fluency I described is a significant asset to advancing the state of the art in CS You're building a cute lil strawman. I think the question is totally out of line with the stated goal. If a college sophomore can answer a question, you're not really assessing much of anything. Also, working at a "hard startup" has nothing to do with "advancing the state of the art in CS". > what do you think a significant asset is? If I'm handling hiring for a "hard startup" and am in search of engineers fit for an "extraordinary team", I'm probably going to spend more time finding applicable skills than opening up to Chapter 1 in the closest algorithms book.
- jimbob45 5y ago
  https://twitter.com/mxcl/status/608682016205344768?lang=en https://twitter.com/mxcl/status/608682016205344768?lang=en Rigid interviewing techniques like that can easily weed out great candidates. Your confidence in your own process leads me to believe you may have missed a few great ones yourself.
  - Hermitian909 5y ago
    There are a lot more bad candidates than great ones, time is limited so it's usually more important to be able to weed out the bad ones fast. Max gets brought up in these discussions a lot, but honestly I think the no hire was a good decision from what I know. Homebrew was a triumph of product design more than technical prowess, I suspect Google might have hired him as a product manager. Max was by his own admission was not a great programmer[0] but interviewed solely for a software role. I don't have a CS degree either, but I wouldn't hire someone to work google scale software if they couldn't invert a binary tree. https://www.quora.com/Whats-the-logic-behind-Google-rejecting-Max-Howell-the-author-of-Homebrew-for-not-being-able-to-invert-a-binary-tree https://www.quora.com/Whats-the-logic-behind-Google-rejectin...
  - qPM9l3XJrF 5y ago
    I don't feel very confident in my process overall and I agree this test will produce a lot of false negatives (and said as much in my comment). But, I've never seen someone who had that sort of extraordinary performance on a leetcode style problem go on to bomb the rest of the interview. These extraordinary performers have pretty much always been solid in other areas as well.
Hermitian909 5y ago
> has essentially created a test for people who have spent a lot of time in competitive programming, or who spent a lot of time on interview prep. He's created a test for people like this or really smart people and in fact both tend to be fairly good hires. If they're smart, they're smart, and if not they've demonstrated high conscientiousness (the psych term for being organized and hard working). I'm not saying 8 hour leetcode interviews are a good idea, but one challenging toy problem like this gives decent signal that the candidate is at least smart or hard working (maybe both) which is more than many candidates.
- me_me_me 5y ago
  Code interviews should not test if you can write a correct program but if you can design and breakdown a problem + discuss possible solutions with the interviewers. That's what programmers do, that's what you should check if they can do. Not if you can one liner some assembly to write hash into db that will trigger an event in main application all in one go.

qPM9l3XJrF 5y ago

Does the author claim this isn't something you can train for? I've given hundreds of coding interviews. I always timed the candidate (with their knowledge). So I have a general idea of the distribution of programming speed, and my guess for this particular problem is to solve it in 10 minutes, you need both talent and practice... just one isn't enough. So asking people to solve it in 10 minutes ends up filtering out untalented people who have practiced a lot (and talented people who haven't practiced as much)

void_mint 5y ago
I'd much rather my engineers have spent their time practicing their actual jobs and not brainteaser interviews. To wit, the interview questions I ask are usually in line with the role they're assuming.
- qPM9l3XJrF 5y ago
  The author wrote this post in the context of hiring at a "hard technology startup". I interpret that as a place where engineers are expected to be able to do cutting edge computer science research (it's not a "well-established [role] to do specialized tasks"). It's not clear to me what sort of questions you would ask "in line with the role they're assuming" for this scenario. One idea here is to give candidates an open problem in CS and ask them to solve it. Which is a nice idea, but one of the other things I learned through administering hundreds of interviews is that open-ended questions aren't great interview problems because they depend a lot on creativity/lateral thinking/insight, which highly benefits from being in a relaxed frame of mind. So these questions end up being a test of how relaxed the candidate is. Stress creates tunnel vision, which is terrible for generating interesting research ideas but it's OK for solving these kind of leetcode problems. So checking for a very high level of programming aptitude, as a proxy for CS research ability, is an approach which is a bit more fair to candidates who are stressed out by interviews. (I also think it is a pretty decent proxy, because doing great CS research requires you to quickly & fluidly generate & evaluate algorithms / data structures which might solve your problem, which is a big part of what a great leetcoder does.) If you're targeting a very high level of generic programming aptitude, it is arguably most fair to make use of a standard method of measuring it. Leetcode problems are the industry standard for measuring programming aptitude. People know to practice them a lot and they know what to expect. If you came up with your own unique way to measure programming aptitude, that would create an even greater burden on candidates to practice (they'd have to do a whole different sort of practice in order to succeed in your interview), and also create anxiety due to an unexpected interview format. I think a lot of readers are overreacting because they didn't pay enough attention to this bit: "It's applicable if you're building an extraordinary team at a hard technology startup." The vast majority of companies in SV are not doing hard technology and don't need an extraordinary team. People should not feel inadequate if they aren't capable of improving the state of the art in technical areas of CS such as databases. This is ordinarily the domain of PhDs, and filtering for demonstrated algorithmic aptitude (as opposed to academic credentials) is actually a pretty egalitarian approach.
  - void_mint 5y ago
    So let's be clear. You're honing in on this block: > "It's applicable if you're building an extraordinary team at a hard technology startup." And you're accepting that a timed question about tic tac toe is enough to prove you're capable of being on an "extraordinary team" at a "hard technology startup"? Really?
    - qPM9l3XJrF 5y ago
      Yes, I think to a large degree being able to build hard technology is a matter of being extremely fluent with writing code and reasoning about data structures and algorithms, as I stated. Being able to solve this problem in 10 minutes seems like a hard-to-fake demonstration of such fluency. When I think about CS open problems and when I solve leetcode problems with algorithmic content to them like this one, it feels like I'm using the same part of my brain (or at least there is significant overlap). If you don't agree with me that the fluency I described is a significant asset to advancing the state of the art in CS, what do you think a significant asset is?
      - void_mint 5y ago
        > Yes, I think to a large degree being able to build hard technology is a matter of being extremely fluent with writing code and reasoning about data structures and algorithms Sure. Again. I don't really think an algorithm that shows up as an introduction to algorithm proves much more than a person read "Intro to Algorithms". So again, a timed introductory problem proves some elite technical skill? > If you don't agree with me that the fluency I described is a significant asset to advancing the state of the art in CS You're building a cute lil strawman. I think the question is totally out of line with the stated goal. If a college sophomore can answer a question, you're not really assessing much of anything. Also, working at a "hard startup" has nothing to do with "advancing the state of the art in CS". > what do you think a significant asset is? If I'm handling hiring for a "hard startup" and am in search of engineers fit for an "extraordinary team", I'm probably going to spend more time finding applicable skills than opening up to Chapter 1 in the closest algorithms book.
jimbob45 5y ago
https://twitter.com/mxcl/status/608682016205344768?lang=en https://twitter.com/mxcl/status/608682016205344768?lang=en Rigid interviewing techniques like that can easily weed out great candidates. Your confidence in your own process leads me to believe you may have missed a few great ones yourself.
- Hermitian909 5y ago
  There are a lot more bad candidates than great ones, time is limited so it's usually more important to be able to weed out the bad ones fast. Max gets brought up in these discussions a lot, but honestly I think the no hire was a good decision from what I know. Homebrew was a triumph of product design more than technical prowess, I suspect Google might have hired him as a product manager. Max was by his own admission was not a great programmer[0] but interviewed solely for a software role. I don't have a CS degree either, but I wouldn't hire someone to work google scale software if they couldn't invert a binary tree. https://www.quora.com/Whats-the-logic-behind-Google-rejecting-Max-Howell-the-author-of-Homebrew-for-not-being-able-to-invert-a-binary-tree https://www.quora.com/Whats-the-logic-behind-Google-rejectin...
- qPM9l3XJrF 5y ago
  I don't feel very confident in my process overall and I agree this test will produce a lot of false negatives (and said as much in my comment). But, I've never seen someone who had that sort of extraordinary performance on a leetcode style problem go on to bomb the rest of the interview. These extraordinary performers have pretty much always been solid in other areas as well.

Hermitian909 5y ago

> has essentially created a test for people who have spent a lot of time in competitive programming, or who spent a lot of time on interview prep. He's created a test for people like this or really smart people and in fact both tend to be fairly good hires. If they're smart, they're smart, and if not they've demonstrated high conscientiousness (the psych term for being organized and hard working). I'm not saying 8 hour leetcode interviews are a good idea, but one challenging toy problem like this gives decent signal that the candidate is at least smart or hard working (maybe both) which is more than many candidates.

me_me_me 5y ago
Code interviews should not test if you can write a correct program but if you can design and breakdown a problem + discuss possible solutions with the interviewers. That's what programmers do, that's what you should check if they can do. Not if you can one liner some assembly to write hash into db that will trigger an event in main application all in one go.