4 ms·

Who asks programming puzzles anymore? When was the last time you applied for a software developer job?

tptacek 13y ago
Before Matasano, if you don't count clients interviewing us. Who asks programming puzzles? Read interview postmortems. The answer appears to be "everyone". We are probably just talking past each other: asking someone how to sanitize an input or parse an HTML document during a face-to-face interview is a "programming puzzle".
- balls187 13y ago
  It's hard to imagine that Facebook, Twitter, Dropbox, Google, Amazon, Apple, and all the rest of the companies that hire software developers, who have more or less the same type of interview, have gotten things wrong, yet have been so successful in hiring good people and building good products. Perhaps the real problem is that those companies have taken all the people who perform well at whiteboard interviews, and what's left are people who struggle. Still, I wouldn't say the system is flawed because it selects a certain type of people. Clearly those people do well.
  - tptacek 13y ago
    It's not at all hard to imagine that the best known company in the industry, the one that spends the most on direct outreach to clients, whose compensation system more or less sets the standard for the industry, a company that is essentially rolling in money, is able to make a conventional recruiting process viable for itself. Having said that, if you talk to Google people who are close to hiring, especially about the process that connects "interviewers" to "recruiters", I don't think you're going to hear consistently good things about how the system works in practice.
  - benched 13y ago
    The point you are grazing is that this approach results in many false negatives, and few false positives. This is workable for companies that are consistently able to attract large numbers of candidates, such as the ones you listed. (And yes, there are false positives. I personally witnessed someone at a large company who was very adept at whiteboard coding, and a very lazy employee.)
    - balls187 13y ago
      You've said it more eloquently than I can, but I want to bring up that: "results in many false negatives..." is not some universal truth. Is there evidence that shows current software interviews turn down way too many great candidates? There should exist a large enough group of developers who have been rejected a number of times[0], who have gone on to be very successful[1] in their careers. Does this data exist? I agree that the current process isn't perfect, and it does produce some false negatives and some false positives. The amount which, has yet to deter me from beliving that the system works, as intended. Certainly there are some fringe candidates, who are turned away, but I don't think it's as onerous as people are making it out to be. Also, most companies didn't start out with unlimited piles of cash, nor could they have their pick of employees. They had to work at it, and I don't believe the culture of their hiring has changed significantly. For the record, I struggle with traditional coding puzzle interviews. But I also know coding on the whiteboard is a skill, which can be practiced, and with time, improved upon, such that if you are a really good developer, but struggle with passing interviews, you can deal with that problem by becoming better at interviewing. Or you know, create something awesome, and never have to worry about interviewing again. [0] Failing one or two interviews isn't really indicative of anything. So this group would require consistently failing multiple standard coding interviews. [1] How to measure success? I say gone to be a major contributor to the success of a company, or built their own company that is considered "successful."
      - lstamour 13y ago
        I'd add to this that either way, whether interviews work or not, the problem is more complex than that: is the potential hire at the right point in his or her life to work for you? Are current open positions a good fit? Will the team get along? This may speak more about less mature candidates, or for smaller companies with more specific skill requirements, but my ultimate point is that rejection shouldn't be forever but few companies take that route...
      - nilkn 13y ago
        Your post seems to employ circular logic. You're asking for evidence of one of your assumptions: > Failing one or two interviews isn't really indicative of anything.
  - ashayh 13y ago
    Yet, these large companies are always desperate to aqcuire, or 'acq-hire' smaller companies.
  - chrisbennet 13y ago
    Googled used to do the puzzle thing and gather copious metrics on how it worked out- it turned out that it didn't. http://qz.com/96206/google-admits-those-infamous-brainteasers-were-completely-useless-for-hiring/ http://qz.com/96206/google-admits-those-infamous-brainteaser...
    - lstamour 13y ago
      Note the word programming. If a brainteaser puzzle has nothing to do with the job you'll be performing then of course it won't indicate real-world performance as well as one that's related to the opening at hand.
  - ken47 13y ago
    It's easy to imagine that smart people's thought processes are imprisoned by their own egos. It is an easy trap for a smart person to fall into. Algorithmic testing has real practical implication if the role requires it, which a subset of the engineers at the aforementioned companies will use. The other practical implication is standardization of the interview process, rooted around an academic background in computer science. It's easy to test whether someone knows a "fundamental" (whether it's actually fundamental to the job is another question) cs data structure or algorithm. The idea is, interview for square pegs, since it is easier to measure the quality of that particular shape at scale. If there is a high-quality peg that happens not to be a square, it is a loss that the system is willing to accept, provided that a large enough % of the good candidates are squares.
    - balls187 13y ago
      Great points. Though I believe algorithmic knowledge is a lot more practical than people give it credit for. I'm not talking about crazy graph searches, or implementing A* in ASM. Are you going to be efficiently searching strings, or reordering arrays, full time? Probably not, but you will likely come across a time where you will need to think about the consequences of your design, even in a simple crud web-app. And that stuff is important.
  - gclaramunt 13y ago
    > It's hard to imagine that Facebook, Twitter, Dropbox, Google, Amazon, Apple, and all the rest of the companies that hire software developers, who have more or less the same type of interview, have gotten things wrong, yet have been so successful in hiring good people and building good products. It could be also cargo cult. If you are a startup, how you will add a technical founder or a first technical hire? Just by his/her ability to write O(log n) searchs in a whiteboard or you will dig deeper and make sure is the perfect match?
    - balls187 13y ago
      Hopefully the startup will dig deeper. Early in the life of your company you're hiring for very long term, and you need generalists. Unless your company is building a product around whiteboard coding or something.
- baddox 13y ago
  I've had to do a bit of sanitizing input and parsing HTML documents "for real" at my job. It's not a primary duty, of course, but it comes up now and then. It seems like a reasonable test for "can they actually write some code to a well-defined spec," although probably not a great test for "are they good at working on a team and architecting bigger projects."
  - nhangen 13y ago
    This is also a practice that is extremely easy to teach and fix.
    - balls187 13y ago
      But can be very costly to your business. SQL Injection--not html parsing ;)

Before Matasano, if you don't count clients interviewing us. Who asks programming puzzles? Read interview postmortems. The answer appears to be "everyone". We are probably just talking past each other: asking someone how to sanitize an input or parse an HTML document during a face-to-face interview is a "programming puzzle".

balls187 13y ago
It's hard to imagine that Facebook, Twitter, Dropbox, Google, Amazon, Apple, and all the rest of the companies that hire software developers, who have more or less the same type of interview, have gotten things wrong, yet have been so successful in hiring good people and building good products. Perhaps the real problem is that those companies have taken all the people who perform well at whiteboard interviews, and what's left are people who struggle. Still, I wouldn't say the system is flawed because it selects a certain type of people. Clearly those people do well.
- tptacek 13y ago
  It's not at all hard to imagine that the best known company in the industry, the one that spends the most on direct outreach to clients, whose compensation system more or less sets the standard for the industry, a company that is essentially rolling in money, is able to make a conventional recruiting process viable for itself. Having said that, if you talk to Google people who are close to hiring, especially about the process that connects "interviewers" to "recruiters", I don't think you're going to hear consistently good things about how the system works in practice.
- benched 13y ago
  The point you are grazing is that this approach results in many false negatives, and few false positives. This is workable for companies that are consistently able to attract large numbers of candidates, such as the ones you listed. (And yes, there are false positives. I personally witnessed someone at a large company who was very adept at whiteboard coding, and a very lazy employee.)
  - balls187 13y ago
    You've said it more eloquently than I can, but I want to bring up that: "results in many false negatives..." is not some universal truth. Is there evidence that shows current software interviews turn down way too many great candidates? There should exist a large enough group of developers who have been rejected a number of times[0], who have gone on to be very successful[1] in their careers. Does this data exist? I agree that the current process isn't perfect, and it does produce some false negatives and some false positives. The amount which, has yet to deter me from beliving that the system works, as intended. Certainly there are some fringe candidates, who are turned away, but I don't think it's as onerous as people are making it out to be. Also, most companies didn't start out with unlimited piles of cash, nor could they have their pick of employees. They had to work at it, and I don't believe the culture of their hiring has changed significantly. For the record, I struggle with traditional coding puzzle interviews. But I also know coding on the whiteboard is a skill, which can be practiced, and with time, improved upon, such that if you are a really good developer, but struggle with passing interviews, you can deal with that problem by becoming better at interviewing. Or you know, create something awesome, and never have to worry about interviewing again. [0] Failing one or two interviews isn't really indicative of anything. So this group would require consistently failing multiple standard coding interviews. [1] How to measure success? I say gone to be a major contributor to the success of a company, or built their own company that is considered "successful."
    - lstamour 13y ago
      I'd add to this that either way, whether interviews work or not, the problem is more complex than that: is the potential hire at the right point in his or her life to work for you? Are current open positions a good fit? Will the team get along? This may speak more about less mature candidates, or for smaller companies with more specific skill requirements, but my ultimate point is that rejection shouldn't be forever but few companies take that route...
    - nilkn 13y ago
      Your post seems to employ circular logic. You're asking for evidence of one of your assumptions: > Failing one or two interviews isn't really indicative of anything.
- ashayh 13y ago
  Yet, these large companies are always desperate to aqcuire, or 'acq-hire' smaller companies.
- chrisbennet 13y ago
  Googled used to do the puzzle thing and gather copious metrics on how it worked out- it turned out that it didn't. http://qz.com/96206/google-admits-those-infamous-brainteasers-were-completely-useless-for-hiring/ http://qz.com/96206/google-admits-those-infamous-brainteaser...
  - lstamour 13y ago
    Note the word programming. If a brainteaser puzzle has nothing to do with the job you'll be performing then of course it won't indicate real-world performance as well as one that's related to the opening at hand.
- ken47 13y ago
  It's easy to imagine that smart people's thought processes are imprisoned by their own egos. It is an easy trap for a smart person to fall into. Algorithmic testing has real practical implication if the role requires it, which a subset of the engineers at the aforementioned companies will use. The other practical implication is standardization of the interview process, rooted around an academic background in computer science. It's easy to test whether someone knows a "fundamental" (whether it's actually fundamental to the job is another question) cs data structure or algorithm. The idea is, interview for square pegs, since it is easier to measure the quality of that particular shape at scale. If there is a high-quality peg that happens not to be a square, it is a loss that the system is willing to accept, provided that a large enough % of the good candidates are squares.
  - balls187 13y ago
    Great points. Though I believe algorithmic knowledge is a lot more practical than people give it credit for. I'm not talking about crazy graph searches, or implementing A* in ASM. Are you going to be efficiently searching strings, or reordering arrays, full time? Probably not, but you will likely come across a time where you will need to think about the consequences of your design, even in a simple crud web-app. And that stuff is important.
- gclaramunt 13y ago
  > It's hard to imagine that Facebook, Twitter, Dropbox, Google, Amazon, Apple, and all the rest of the companies that hire software developers, who have more or less the same type of interview, have gotten things wrong, yet have been so successful in hiring good people and building good products. It could be also cargo cult. If you are a startup, how you will add a technical founder or a first technical hire? Just by his/her ability to write O(log n) searchs in a whiteboard or you will dig deeper and make sure is the perfect match?
  - balls187 13y ago
    Hopefully the startup will dig deeper. Early in the life of your company you're hiring for very long term, and you need generalists. Unless your company is building a product around whiteboard coding or something.
baddox 13y ago
I've had to do a bit of sanitizing input and parsing HTML documents "for real" at my job. It's not a primary duty, of course, but it comes up now and then. It seems like a reasonable test for "can they actually write some code to a well-defined spec," although probably not a great test for "are they good at working on a team and architecting bigger projects."
- nhangen 13y ago
  This is also a practice that is extremely easy to teach and fix.
  - balls187 13y ago
    But can be very costly to your business. SQL Injection--not html parsing ;)