6 ms·

I think the major disconnect here is in the industry's general inability to recognize ability through the standard interview processes. On one hand you have th

by jwoah12 11y ago

I think the major disconnect here is in the industry's general inability to recognize ability through the standard interview processes. On one hand you have the common whiteboard coding interviews popular among the big tech players (and much of the rest of the industry). I'm definitely biased here, as someone who is average/below average at these types of interviews but an above average (I hope) developer, but I don't think these interviews are very good indicators of on-the-job performance. On the other hand you have many startups who won't even interview you if you don't have side projects or open source contributions to show. As someone who has side projects to show, I understand that most people do not, and it's unrealistic to expect everyone to. The best way I can think of to predict how someone will perform is to have them do something relatively close to what they'd actually be doing, like going over some of your actual code with them and asking them to talk about what it's doing, and doing some pair programming with them on an actual computer. Has anyone had success with something like that?

rco8786 11y ago
Lots of companies do that, fwiw...though I agree that many companies may benefit from doing it more instead of blindly following the "big tech" whiteboard style. Here's the thing about the whiteboard coding interviews: A) There's a huge false negative rate. I.E. a lot of good engineers get turned down because of a bad whiteboard exercise. B) The "big tech" companies know this and do not care. They have people who want to work there lined up out the door. The whiteboard interview is setup to minimize false positives. C) The above argument can possibly be made in favor of the pair programming/"do what you'll actually be doing" type of interview and with even less false negatives, but frankly those interviews are harder and more time consuming on the interviewer(s). Additionally, the whiteboard panel gives 5-6 people time to meet and speak with the candidate instead of 1-2 which helps to mitigate the effects of any personal biases that might be present. Tl;Dr - the whiteboard interviews actually work well when you have a giant pool of potentials and only optimize for minimizing false positives.
- nulltype 11y ago
  Part B there is why I suspect this tech talent shortage is not as real as it's made out to be. If there was a shortage, they would totally care.
  - rco8786 11y ago
    I don't know if it's necessarily mutually exclusive. It's only a handful of company's that can afford to do this sort of interviewing. Judging by the amount of recruiter activity in my email/LinkedIn I'd say there's plenty of non "big tech" companies out there that are really struggling to find people.
  - jdmichal 11y ago
    Just because I have enough food on my table does not mean that no one is starving.
- jdmichal 11y ago
  > The "big tech" companies know this and do not care. They have people who want to work there lined up out the door. Also importantly, the big companies seem to purposefully have a short memory to help counteract this. The chances you even interview in the same division are relatively low, much less with the same team. So why not let people try again and again and again? If they really are good, they'll eventually find a fit and get in.
  - rco8786 11y ago
    Yep, I made note of that in my OP as well :) edit oops, no I didn't. I had a note about it originally but deleted before posting. Either way, that is correct.
- jwoah12 11y ago
  Totally agree that the aim is to minimize false positives. although I think you could devise the pair programming type interview to still allow for 1 hour chunks with many different people doing the sessions. A good phone screen should be able to filter out candidates who obviously don't have the foundation, and watching them code in a (semi)realistic environment can tease out things like resourcefulness, search IQ, etc. I find those things to be better indicators than the ability to find the distance between two nodes in a binary tree.
  - rco8786 11y ago
    > I think you could devise the pair programming type interview to still allow for 1 hour chunks I try and do my interviews this way. While I am required to use the whiteboard and the interviewee is required to write on it, I try to make it as clear as possible that we can brainstorm/strategize on the problem together. Of course I've asked the same few questions probably 100 times now and have seen them solved every which way so I have to play a little dumb sometimes. Same problem occurs in a "real" pair programming environment too, though. Interviewing is hard, and it's easy to throw stones at companies who don't do it the way you want it done.
- vonmoltke 11y ago
  > The whiteboard interview is setup to minimize false positives. I have yet to see evidence that it actually does this. Everything I have seen so far is of the form, "$BIG_COMPANY does whiteboard interviews, $BIG_COMPANY hires few bad people, therefore whiteboard interviews minimize false positives". The problem is, that says nothing about the effect the whiteboard interview has on doing this. Would the rate change if it were removed? Would the change make the rate worse or better?
  - thedufer 11y ago
    Anecdotally, we switched up our process some in an attempt to be able to hire faster (mostly by avoiding false negatives). It used to be standard whiteboard style; we switched to a mix of that and more real-work style interviews. False positives went up pretty significantly, and even after shedding those the general quality was of hires was a bit lower.
    - Alyoshi 11y ago
      How long did you keep the new practices before reverting? Any company making this change could run into the issue of interviewers being well practiced at getting meaningful information out of whiteboard interviews and inexperienced at getting meaningful information out of real-work style interviews. Additionally, hiring successful whiteboard interviewees may actually select for people who are better at getting meaningful information out of whiteboard interviews and in turn perpetuate the relative effectiveness of solutions that are less effective in absolute. Which would bring us back to the problem being one of company culture. And this is a case where it is particularly expensive to fix it.
      - thedufer 11y ago
        We didn't revert. I've since left, but they're still using the newer method. I believe the decrease in time to fill positions was determined to be worth the decrease in quality. It's been over a year - maybe 2.
- mellery451 11y ago
  it's the false positives that worry me more - that is, the well-practiced whiteboard coder who cannot actually design a maintainable system or the sociopath or brilliant jerk who nobody can stand to work with.
- tokipin 11y ago
  it's a simple idea in theory, but the reality of these things is a different beast, in particular that these tests form a filter not just in terms of degree, but also in terms of kind. this is why Google is Google and is extreme in both its strengths and weaknesses. what i'm curious about is how Apple weathers these particular waters, since they have more to lose than any of the other big companies ever had. hehehe.
cletus 11y ago
I've had conversations with various people about startups to "solve hiring". Thing is, I don't think it's solvable any more than another engineer's trap: fixing online dating. People are involved and whether or not someone will work out is no more deterministic than figuring out if two people dating would result in a long term relationship. The biggest problem with coding whiteboard problems is engineers fall into the trap of thinking the problem needs to be hard, which actually reduces the value of the test. A coding test is simply to weed out idiots. That's why FizzBuzz is a simple problem. It's a negative filter. Acing such an easy test doesn't mean you'll be an awesome engineer but failing an easy problem almost certainly means you won't be an awesome engineer. Substitute with a hard problem and a pass signal doesn't get much better but a fail signal gets far noisier and you're going to get an awful lot more false negatives.
citizens 11y ago
> "On the other hand you have many startups who won't even interview you if you don't have side projects or open source contributions to show" In my experience having too many side projects can be perceived as a negative for some employers.
- solutionyogi 11y ago
  I hear you however that's shortsighted on part of the employer. As an employer, you have a projects/tasks which needs to be done, you set up estimates on how long it should take. If the employee does it within those estimates, what he/she does in their spare time should not matter at all. Personally, I would love to hire a person who is doing side projects to keep learning. I feel very confident that they will be able to take care of the regular tasks assigned as part of the job.
- J-dawg 11y ago
  Why would too many side projects be perceived as negative? I'm genuinely struggling to think of a reason.
  - CaptSpify 11y ago
    I can think of two (note: I don't think they are valid). A) You won't have time to work "outside of office hours". A lot of tech is notorious for having people work all kinds of hours. If your busy working on side-projects, you won't be working on work-projects B) Burn out. If you spend all your time coding, you might end up burning out after the company has invested money and time into you.

by jwoah12 11y ago

rco8786 11y ago

Lots of companies do that, fwiw...though I agree that many companies may benefit from doing it more instead of blindly following the "big tech" whiteboard style. Here's the thing about the whiteboard coding interviews: A) There's a huge false negative rate. I.E. a lot of good engineers get turned down because of a bad whiteboard exercise. B) The "big tech" companies know this and do not care. They have people who want to work there lined up out the door. The whiteboard interview is setup to minimize false positives. C) The above argument can possibly be made in favor of the pair programming/"do what you'll actually be doing" type of interview and with even less false negatives, but frankly those interviews are harder and more time consuming on the interviewer(s). Additionally, the whiteboard panel gives 5-6 people time to meet and speak with the candidate instead of 1-2 which helps to mitigate the effects of any personal biases that might be present. Tl;Dr - the whiteboard interviews actually work well when you have a giant pool of potentials and only optimize for minimizing false positives.

nulltype 11y ago
Part B there is why I suspect this tech talent shortage is not as real as it's made out to be. If there was a shortage, they would totally care.
- rco8786 11y ago
  I don't know if it's necessarily mutually exclusive. It's only a handful of company's that can afford to do this sort of interviewing. Judging by the amount of recruiter activity in my email/LinkedIn I'd say there's plenty of non "big tech" companies out there that are really struggling to find people.
- jdmichal 11y ago
  Just because I have enough food on my table does not mean that no one is starving.
jdmichal 11y ago
> The "big tech" companies know this and do not care. They have people who want to work there lined up out the door. Also importantly, the big companies seem to purposefully have a short memory to help counteract this. The chances you even interview in the same division are relatively low, much less with the same team. So why not let people try again and again and again? If they really are good, they'll eventually find a fit and get in.
- rco8786 11y ago
  Yep, I made note of that in my OP as well :) edit oops, no I didn't. I had a note about it originally but deleted before posting. Either way, that is correct.
jwoah12 11y ago
Totally agree that the aim is to minimize false positives. although I think you could devise the pair programming type interview to still allow for 1 hour chunks with many different people doing the sessions. A good phone screen should be able to filter out candidates who obviously don't have the foundation, and watching them code in a (semi)realistic environment can tease out things like resourcefulness, search IQ, etc. I find those things to be better indicators than the ability to find the distance between two nodes in a binary tree.
- rco8786 11y ago
  > I think you could devise the pair programming type interview to still allow for 1 hour chunks I try and do my interviews this way. While I am required to use the whiteboard and the interviewee is required to write on it, I try to make it as clear as possible that we can brainstorm/strategize on the problem together. Of course I've asked the same few questions probably 100 times now and have seen them solved every which way so I have to play a little dumb sometimes. Same problem occurs in a "real" pair programming environment too, though. Interviewing is hard, and it's easy to throw stones at companies who don't do it the way you want it done.
vonmoltke 11y ago
> The whiteboard interview is setup to minimize false positives. I have yet to see evidence that it actually does this. Everything I have seen so far is of the form, "$BIG_COMPANY does whiteboard interviews, $BIG_COMPANY hires few bad people, therefore whiteboard interviews minimize false positives". The problem is, that says nothing about the effect the whiteboard interview has on doing this. Would the rate change if it were removed? Would the change make the rate worse or better?
- thedufer 11y ago
  Anecdotally, we switched up our process some in an attempt to be able to hire faster (mostly by avoiding false negatives). It used to be standard whiteboard style; we switched to a mix of that and more real-work style interviews. False positives went up pretty significantly, and even after shedding those the general quality was of hires was a bit lower.
  - Alyoshi 11y ago
    How long did you keep the new practices before reverting? Any company making this change could run into the issue of interviewers being well practiced at getting meaningful information out of whiteboard interviews and inexperienced at getting meaningful information out of real-work style interviews. Additionally, hiring successful whiteboard interviewees may actually select for people who are better at getting meaningful information out of whiteboard interviews and in turn perpetuate the relative effectiveness of solutions that are less effective in absolute. Which would bring us back to the problem being one of company culture. And this is a case where it is particularly expensive to fix it.
    - thedufer 11y ago
      We didn't revert. I've since left, but they're still using the newer method. I believe the decrease in time to fill positions was determined to be worth the decrease in quality. It's been over a year - maybe 2.
mellery451 11y ago
it's the false positives that worry me more - that is, the well-practiced whiteboard coder who cannot actually design a maintainable system or the sociopath or brilliant jerk who nobody can stand to work with.
tokipin 11y ago
it's a simple idea in theory, but the reality of these things is a different beast, in particular that these tests form a filter not just in terms of degree, but also in terms of kind. this is why Google is Google and is extreme in both its strengths and weaknesses. what i'm curious about is how Apple weathers these particular waters, since they have more to lose than any of the other big companies ever had. hehehe.

cletus 11y ago

I've had conversations with various people about startups to "solve hiring". Thing is, I don't think it's solvable any more than another engineer's trap: fixing online dating. People are involved and whether or not someone will work out is no more deterministic than figuring out if two people dating would result in a long term relationship. The biggest problem with coding whiteboard problems is engineers fall into the trap of thinking the problem needs to be hard, which actually reduces the value of the test. A coding test is simply to weed out idiots. That's why FizzBuzz is a simple problem. It's a negative filter. Acing such an easy test doesn't mean you'll be an awesome engineer but failing an easy problem almost certainly means you won't be an awesome engineer. Substitute with a hard problem and a pass signal doesn't get much better but a fail signal gets far noisier and you're going to get an awful lot more false negatives.

citizens 11y ago

> "On the other hand you have many startups who won't even interview you if you don't have side projects or open source contributions to show" In my experience having too many side projects can be perceived as a negative for some employers.

solutionyogi 11y ago
I hear you however that's shortsighted on part of the employer. As an employer, you have a projects/tasks which needs to be done, you set up estimates on how long it should take. If the employee does it within those estimates, what he/she does in their spare time should not matter at all. Personally, I would love to hire a person who is doing side projects to keep learning. I feel very confident that they will be able to take care of the regular tasks assigned as part of the job.
J-dawg 11y ago
Why would too many side projects be perceived as negative? I'm genuinely struggling to think of a reason.
- CaptSpify 11y ago
  I can think of two (note: I don't think they are valid). A) You won't have time to work "outside of office hours". A lot of tech is notorious for having people work all kinds of hours. If your busy working on side-projects, you won't be working on work-projects B) Burn out. If you spend all your time coding, you might end up burning out after the company has invested money and time into you.