4 ms·

Finally, some validation that my interpretation of their process is not insane. One thing that really rustles my jimmies is the constant assertion that "false

Finally, some validation that my interpretation of their process is not insane. One thing that really rustles my jimmies is the constant assertion that "false negatives are (effectively) free". I think Google and the companies who hire like them seriously underestimate how much this costs them, both the direct costs of spending so much to ultimately reject people and the indirect costs from the work that is not getting done or being foisted on another overloaded engineer.

JoeAltmaier 11y ago
The root issue is that, the costs of bad hiring are even higher than that. For a rapidly-growing company bad hires can snowball into a bad organization.
- tptacek 11y ago
  I agree, and that's even more reason that companies should use work sample tests in preference to on-site coding interviews.
- vonmoltke 11y ago
  Apparently this is where I will disagree with Tom, because I think the costs of a bad hire are often overstated. Part of it is loss aversion, and part of it is inability to quantify risk. In fact, it seems like the industry is at a point right now where they would rather repeat this mantra than put effort into understanding the scope of the problem.
  - JoeAltmaier 11y ago
    The costs of a bad hire are overstated. The cost of systemic bad hiring may grow geometrically. If that estimation is correct (bad hires result in more bad hires) then no price is too much to avoid a bad hire. And consider the US Air Force. They have an unlimited number of young people willing to learn to fly jets. Qualified candidates even. So they can filter any way they like, and still not run out of applicants. So their filters seem capricious, because they sometimes are. And it doesn't matter. Is Silicon Valley in that position? Depends upon who you are. I don't think Google for instance is running out of candidates.
- GVIrish 11y ago
  I suspect that the idea that the costs of bad hires are huge comes from not identifying bad hires early. If you hire a jerk who makes poor coding decisions and fail to oust him for a year or more, the effect could be devastating. If said jerk is spotted in a month or so, it may not be so costly. And really there are levels of bad hire. There's the bad hire who really doesn't know how to do the job and will never get good enough to do it. That person should be very easy to spot with not too much effort in the interview process. The dangerous hire is someone who on the surface can do the work but has a toxic attitude and/or doesn't learn/grow. I'm not sure putting someone through a torturous interview process is going to root that person out. I think where a lot of the elitist hiring is at now is that many companies aren't so much trying to filter out people who can't do the job, so much as they're holding out for who they think are rockstars. So they put candidates through the ringer with the thought that what will be left is rockstar material. But many highly-qualified individuals won't jump through hoops for any but the top tier companies, and sometimes not even then. Furthermore, by definition, rockstars make up a very small percentage of the devs out there. What are the chances that every company who is holding out for elite coders even has any applying? Especially considering that elite people probably don't jump around often.
tptacek 11y ago
I worked in a small microcosm of Google's market (Google wants the "best of the best" software developers, and we wanted qualified app pentesters; both are a small subset of the overall market for software talent). My experience is that running a hiring process that lights up only on the kind of talent that qualifies itself with a standard tech interview is ludicrously expensive. People that do well in standard tech interviews can work anywhere they want. If you can only hire those people, you are competing for talent with the wealthiest (or most overfunded) tech companies in the market. Fortunately: performance in tech interviews is in fact not a good proxy for programming ability (in fact, there are ways in which being good at interviews can obscure deficits in candidates), and you can get ridiculous discounts to the price Google pays for talent if you don't try to hire people the dumb way Google does.

JoeAltmaier 11y ago

The root issue is that, the costs of bad hiring are even higher than that. For a rapidly-growing company bad hires can snowball into a bad organization.

tptacek 11y ago
I agree, and that's even more reason that companies should use work sample tests in preference to on-site coding interviews.
vonmoltke 11y ago
Apparently this is where I will disagree with Tom, because I think the costs of a bad hire are often overstated. Part of it is loss aversion, and part of it is inability to quantify risk. In fact, it seems like the industry is at a point right now where they would rather repeat this mantra than put effort into understanding the scope of the problem.
- JoeAltmaier 11y ago
  The costs of a bad hire are overstated. The cost of systemic bad hiring may grow geometrically. If that estimation is correct (bad hires result in more bad hires) then no price is too much to avoid a bad hire. And consider the US Air Force. They have an unlimited number of young people willing to learn to fly jets. Qualified candidates even. So they can filter any way they like, and still not run out of applicants. So their filters seem capricious, because they sometimes are. And it doesn't matter. Is Silicon Valley in that position? Depends upon who you are. I don't think Google for instance is running out of candidates.
GVIrish 11y ago
I suspect that the idea that the costs of bad hires are huge comes from not identifying bad hires early. If you hire a jerk who makes poor coding decisions and fail to oust him for a year or more, the effect could be devastating. If said jerk is spotted in a month or so, it may not be so costly. And really there are levels of bad hire. There's the bad hire who really doesn't know how to do the job and will never get good enough to do it. That person should be very easy to spot with not too much effort in the interview process. The dangerous hire is someone who on the surface can do the work but has a toxic attitude and/or doesn't learn/grow. I'm not sure putting someone through a torturous interview process is going to root that person out. I think where a lot of the elitist hiring is at now is that many companies aren't so much trying to filter out people who can't do the job, so much as they're holding out for who they think are rockstars. So they put candidates through the ringer with the thought that what will be left is rockstar material. But many highly-qualified individuals won't jump through hoops for any but the top tier companies, and sometimes not even then. Furthermore, by definition, rockstars make up a very small percentage of the devs out there. What are the chances that every company who is holding out for elite coders even has any applying? Especially considering that elite people probably don't jump around often.

tptacek 11y ago

I worked in a small microcosm of Google's market (Google wants the "best of the best" software developers, and we wanted qualified app pentesters; both are a small subset of the overall market for software talent). My experience is that running a hiring process that lights up only on the kind of talent that qualifies itself with a standard tech interview is ludicrously expensive. People that do well in standard tech interviews can work anywhere they want. If you can only hire those people, you are competing for talent with the wealthiest (or most overfunded) tech companies in the market. Fortunately: performance in tech interviews is in fact not a good proxy for programming ability (in fact, there are ways in which being good at interviews can obscure deficits in candidates), and you can get ridiculous discounts to the price Google pays for talent if you don't try to hire people the dumb way Google does.