5 ms·

All this teeth-gnashing regarding programming and logic puzzles are simply to cover up the simple fact that most people (even around here) are insecure about th

All this teeth-gnashing regarding programming and logic puzzles are simply to cover up the simple fact that most people (even around here) are insecure about their programming skill, and more-so their intelligence. The fact is that these types of questions are indeed backed up by the very research you cite, namely that a work-sample test and general mental ability are the only reliable predictors of job performance. But this is exactly what programming puzzles are (and to a lesser extent logic puzzles). A programming puzzle is simply the intersection of a work-sample test and an IQ test. And of course, logic skills and programming skills go hand-in-hand. Instead of writing endless blog posts and internet comments about how useless these tests are (trying to convince a potential future hiring manager not to use them on you), just get better at them! You'll be better off for it. Of course, not all programming jobs (most, in fact) need top 10% programmers with gifted IQs. It is important that jobs are honest with themselves regarding the level of talent they need for their next sexting iphone app. But this too is a part of the problem. Everyone believes their shitty app or website is going to change the world, and thus need world-class developers to help them realize their vision. Egos will be the death of us all.

danielweber 14y ago
most people (even around here) are insecure about their programming skill, This has a lot of truth. I think also most people say "the best interview technique is the one I do best at."
- benjaminwootton 14y ago
  I don't think it's anything to with insecurity. I would consider myself a very good programmer compared to the hundreds I've worked directly with - think good quality, clean, well tested code delivered on time that delivers outsized value in production. I am however useless at these mental gymnastic and logic puzzle things. Having a high degree of programming skill is weakly correlated with these kind of interview questions, if that.
ajross 14y ago
I think this is basically true. But a point worth making is that a "puzzle" problem is a much heavier thing than lightweight tests like FizzBuzz. If you hand someone a puzzle you're pretty much guaranteed (absent the rare, truly gifted candidate) to watch them struggle at a whiteboard for half an hour. FizzBuzz (or "reverse words in a sentence", or whatever) will tell you the same thing in a minute or two. Basically, the Venn diagram regions for "can code fluently" and "can solve puzzles" have a ton of overlap. So test the first one, not the second.
- hackinthebochs 14y ago
  You make a good point. The question is how much of an overlap is there? Is there value in adding a component of raw intelligence to the question (say, applying a data structure in a unique and non-obvious way)? It is reasonable to expect that the act of programming fluently entails a sufficient level of IQ such that there is nothing more to be learned from attempting to craft a question that tests intelligence directly? Two bits of information could potentially answer this: does intelligence as a predictor of future job success drop off past a certain level? Also, what does the distribution of IQ look like for those who are fluent coders?
pekk 14y ago
It isn't that people are insecure about their programming skill and intelligence. It's that they are irritated by tests which don't measure what they're supposed to. If any insecurity is involved, it's about ability to ace unrepresentative tests ranging from trivial nonsense to outright hazing. You advise readers to "just get better at [the tests]". That's practical. But it goes right back to the problem, that many of the tests mostly index whether you prepared for that test. Everyone talks about how many applicants are terrible at programming, or how many employees are mediocre. But at least offline, we rarely mention the complementary problem, that many companies are terrible at interviewing and hiring. We talk as if being on the other side of that desk, in a position of power, means you are automatically good at interviewing. It isn't so. When a company treats candidates abusively - or takes bad hires who are extra slick, cheap, submissive, or similar to themselves - that isn't because of bad applicants. It's a failure of that company. We can recognize that a defensive and condescending attitude is toxic in a programmer. But the same attitude in an interviewer is completely accepted. They're on top, applicants are on bottom.
- hackinthebochs 14y ago
  > It's that they are irritated by tests which don't measure what they're supposed to I disagree that programming puzzles don't measure what they're supposed to. Programming puzzles are a microcosm of what programming entails day-to-day. They test the same exact mental faculties. Sure, no one needs to write strcmp using primitives anymore, but the task of doing so exercises the same mental processes that are involved in orchestrating the various components when handling a web request. Doing these type of "low level" tests then allow you to include components that do measure one's mental flexibility and creativity, i.e. intelligence. Sure, we can give a candidate a test that exactly measures what they'll be doing on the job. But what if the job changes? What if we need them to use a language or a framework outside of their comfort zone? This is the reason these tests are not specific to particular tools. You want a programmer that has the intelligence and command of computer science concepts to quickly jump between various tools with little effort. Comparing strings and reversing linked lists do absolutely measure this capability. No, its not testing whether you memorized how to do these things, its testing whether you have the mental flexibility to figure it out on the spot.
derefr 14y ago
> A programming puzzle is simply the intersection of a work-sample test and an IQ test. No: a programming puzzle is the intersection of a work-sample test with one question from an IQ test. IQ tests get a good handle on your intelligence by testing you over a wide range of near-trivial questions, each known to similarly measure intelligence, and then adding the scores up. By giving you so many opportunities to succeed or fail, they don't fall victim to the "I just couldn't figure this one out" problem. Whereas programming puzzles do that all the time. Now, if your interview consisted of 200 programming puzzles, each of which should take roughly 20 seconds to solve, then you could say it's getting a fair handle on someone's IQ as measured through the lens of programming. But throwing two 10-minute puzzles into the interview does nothing to help you measure anything. The "scores" you get are black-and-white threshold filters on a set of numbers (IQ) that are almost totally middle-grey.
- hackinthebochs 14y ago
  > a programming puzzle is the intersection of a work-sample test with one question from an IQ test In the worst case this is true. But ideally the question will have various components, and various levels of difficulty. I'm reminded of a blog posted here some time ago that had the candidate split a string based on a provided dictionary of words (or something to that effect). The best candidates produced maximally efficient code including memoization or dynamic programming. The worst failed to implement a basic "split on the space" functionality. Besides these two extremes there was a whole spectrum in between. The point is to not give questions that are Pass/Fail, but a question that has a spectrum of possible answers allowing one to find the limits of the candidates knowledge.
  - derefr 14y ago
    That still doesn't help the candidates who just don't see the trick. What if you know memoization and dynamic programming, but it just doesn't occur to you that day, that to correctly split on spaces (not adding empty tokens at the beginning and end of the string, etc.) you need to use a state machine? You could give the candidate that "step" of the solution and continue on to see if they know the rest, but the candidate will likely be feeling a loss of confidence. They'll think "hey, if I couldn't answer the 'easier' part, why should I be able to answer the 'hard' part?" and then their mind will lock up even harder when they search it for the next step. This is why IQ tests are organized as a series of orthogonal questions. No question is dependent on any other question, so you can just move on quickly from a failure and prove your worth on another question. In fact, on a usual IQ test, you're even given five or six "presentations" of each question, each slightly different--like math homework questions with different numbers inserted. It happens quite often that you'll get stuck on, say, recognizing how one specific pattern of shapes go together, but will be able to see a different pattern instantly. Interviews do the opposite of this; they are presented as if intelligence is a linear quantity, and if you fail on any part of a puzzle, you couldn't possibly be able to solve a "harder" part. In other words, interviewers are using the "error" theory of intelligence, rather than the "bugs" theory[1]. [1] http://news.ycombinator.com/item?id=5233497 http://news.ycombinator.com/item?id=5233497
doktrin 14y ago
> most people (even around here) are insecure about their programming skill, and more-so their intelligence. > Instead of writing endless blog posts and internet comments about how useless these tests are ... just get better at them! You'll be better off for it. This doesn't hang together. Will getting "better" at programming tests & logic puzzles make you a better programmer? Make you more intelligent? If not, doesn't that illustrate precisely why people seem to have a problem with these tests? Namely, that they are primarily a measure of preparation & studying for the exam?
- hackinthebochs 14y ago
  It depends on your strategy for "getting better at these types of problems". You can cram and memorize all the algorithms you can, and that may allow you to pass some filters that you otherwise would not have. This strategy has its uses: besides being offered more jobs, you now have knowledge of these algorithms in your head and thus more tools in your toolbox. Ideally, the candidate would learn about these "exotic" data structures, understand their properties, learn the cases where they are useful, and most importantly develop mental heuristics to recognize patterns of problems and appropriate classes of solutions. This will also help you pass these filters, and in the process will absolutely make you a much better programmer.
  - doktrin 14y ago
    Fair. Speaking for myself, I have purposely avoided studying specifically for interviews. My reasoning was that by relying on my existing knowledge + logic, I would give my interviewers better insight into my thought process and approach to problem solving. If the benefits of spending time with this material are as significant as you say, perhaps I will revise my strategy (or, lack thereof).

danielweber 14y ago

most people (even around here) are insecure about their programming skill, This has a lot of truth. I think also most people say "the best interview technique is the one I do best at."

benjaminwootton 14y ago
I don't think it's anything to with insecurity. I would consider myself a very good programmer compared to the hundreds I've worked directly with - think good quality, clean, well tested code delivered on time that delivers outsized value in production. I am however useless at these mental gymnastic and logic puzzle things. Having a high degree of programming skill is weakly correlated with these kind of interview questions, if that.

ajross 14y ago

I think this is basically true. But a point worth making is that a "puzzle" problem is a much heavier thing than lightweight tests like FizzBuzz. If you hand someone a puzzle you're pretty much guaranteed (absent the rare, truly gifted candidate) to watch them struggle at a whiteboard for half an hour. FizzBuzz (or "reverse words in a sentence", or whatever) will tell you the same thing in a minute or two. Basically, the Venn diagram regions for "can code fluently" and "can solve puzzles" have a ton of overlap. So test the first one, not the second.

hackinthebochs 14y ago
You make a good point. The question is how much of an overlap is there? Is there value in adding a component of raw intelligence to the question (say, applying a data structure in a unique and non-obvious way)? It is reasonable to expect that the act of programming fluently entails a sufficient level of IQ such that there is nothing more to be learned from attempting to craft a question that tests intelligence directly? Two bits of information could potentially answer this: does intelligence as a predictor of future job success drop off past a certain level? Also, what does the distribution of IQ look like for those who are fluent coders?

pekk 14y ago

It isn't that people are insecure about their programming skill and intelligence. It's that they are irritated by tests which don't measure what they're supposed to. If any insecurity is involved, it's about ability to ace unrepresentative tests ranging from trivial nonsense to outright hazing. You advise readers to "just get better at [the tests]". That's practical. But it goes right back to the problem, that many of the tests mostly index whether you prepared for that test. Everyone talks about how many applicants are terrible at programming, or how many employees are mediocre. But at least offline, we rarely mention the complementary problem, that many companies are terrible at interviewing and hiring. We talk as if being on the other side of that desk, in a position of power, means you are automatically good at interviewing. It isn't so. When a company treats candidates abusively - or takes bad hires who are extra slick, cheap, submissive, or similar to themselves - that isn't because of bad applicants. It's a failure of that company. We can recognize that a defensive and condescending attitude is toxic in a programmer. But the same attitude in an interviewer is completely accepted. They're on top, applicants are on bottom.

hackinthebochs 14y ago
> It's that they are irritated by tests which don't measure what they're supposed to I disagree that programming puzzles don't measure what they're supposed to. Programming puzzles are a microcosm of what programming entails day-to-day. They test the same exact mental faculties. Sure, no one needs to write strcmp using primitives anymore, but the task of doing so exercises the same mental processes that are involved in orchestrating the various components when handling a web request. Doing these type of "low level" tests then allow you to include components that do measure one's mental flexibility and creativity, i.e. intelligence. Sure, we can give a candidate a test that exactly measures what they'll be doing on the job. But what if the job changes? What if we need them to use a language or a framework outside of their comfort zone? This is the reason these tests are not specific to particular tools. You want a programmer that has the intelligence and command of computer science concepts to quickly jump between various tools with little effort. Comparing strings and reversing linked lists do absolutely measure this capability. No, its not testing whether you memorized how to do these things, its testing whether you have the mental flexibility to figure it out on the spot.

derefr 14y ago

> A programming puzzle is simply the intersection of a work-sample test and an IQ test. No: a programming puzzle is the intersection of a work-sample test with one question from an IQ test. IQ tests get a good handle on your intelligence by testing you over a wide range of near-trivial questions, each known to similarly measure intelligence, and then adding the scores up. By giving you so many opportunities to succeed or fail, they don't fall victim to the "I just couldn't figure this one out" problem. Whereas programming puzzles do that all the time. Now, if your interview consisted of 200 programming puzzles, each of which should take roughly 20 seconds to solve, then you could say it's getting a fair handle on someone's IQ as measured through the lens of programming. But throwing two 10-minute puzzles into the interview does nothing to help you measure anything. The "scores" you get are black-and-white threshold filters on a set of numbers (IQ) that are almost totally middle-grey.

hackinthebochs 14y ago
> a programming puzzle is the intersection of a work-sample test with one question from an IQ test In the worst case this is true. But ideally the question will have various components, and various levels of difficulty. I'm reminded of a blog posted here some time ago that had the candidate split a string based on a provided dictionary of words (or something to that effect). The best candidates produced maximally efficient code including memoization or dynamic programming. The worst failed to implement a basic "split on the space" functionality. Besides these two extremes there was a whole spectrum in between. The point is to not give questions that are Pass/Fail, but a question that has a spectrum of possible answers allowing one to find the limits of the candidates knowledge.
- derefr 14y ago
  That still doesn't help the candidates who just don't see the trick. What if you know memoization and dynamic programming, but it just doesn't occur to you that day, that to correctly split on spaces (not adding empty tokens at the beginning and end of the string, etc.) you need to use a state machine? You could give the candidate that "step" of the solution and continue on to see if they know the rest, but the candidate will likely be feeling a loss of confidence. They'll think "hey, if I couldn't answer the 'easier' part, why should I be able to answer the 'hard' part?" and then their mind will lock up even harder when they search it for the next step. This is why IQ tests are organized as a series of orthogonal questions. No question is dependent on any other question, so you can just move on quickly from a failure and prove your worth on another question. In fact, on a usual IQ test, you're even given five or six "presentations" of each question, each slightly different--like math homework questions with different numbers inserted. It happens quite often that you'll get stuck on, say, recognizing how one specific pattern of shapes go together, but will be able to see a different pattern instantly. Interviews do the opposite of this; they are presented as if intelligence is a linear quantity, and if you fail on any part of a puzzle, you couldn't possibly be able to solve a "harder" part. In other words, interviewers are using the "error" theory of intelligence, rather than the "bugs" theory[1]. [1] http://news.ycombinator.com/item?id=5233497 http://news.ycombinator.com/item?id=5233497

doktrin 14y ago

> most people (even around here) are insecure about their programming skill, and more-so their intelligence. > Instead of writing endless blog posts and internet comments about how useless these tests are ... just get better at them! You'll be better off for it. This doesn't hang together. Will getting "better" at programming tests & logic puzzles make you a better programmer? Make you more intelligent? If not, doesn't that illustrate precisely why people seem to have a problem with these tests? Namely, that they are primarily a measure of preparation & studying for the exam?

hackinthebochs 14y ago
It depends on your strategy for "getting better at these types of problems". You can cram and memorize all the algorithms you can, and that may allow you to pass some filters that you otherwise would not have. This strategy has its uses: besides being offered more jobs, you now have knowledge of these algorithms in your head and thus more tools in your toolbox. Ideally, the candidate would learn about these "exotic" data structures, understand their properties, learn the cases where they are useful, and most importantly develop mental heuristics to recognize patterns of problems and appropriate classes of solutions. This will also help you pass these filters, and in the process will absolutely make you a much better programmer.
- doktrin 14y ago
  Fair. Speaking for myself, I have purposely avoided studying specifically for interviews. My reasoning was that by relying on my existing knowledge + logic, I would give my interviewers better insight into my thought process and approach to problem solving. If the benefits of spending time with this material are as significant as you say, perhaps I will revise my strategy (or, lack thereof).