6 ms·

I know how you feel, but take solace in two facts: 1. Any good company will have a high false negative rate, because it's more expensive to hire a bad candidat

by buss 10y ago

I know how you feel, but take solace in two facts: 1. Any good company will have a high false negative rate, because it's more expensive to hire a bad candidate than reject a good one. That's why the big ones (Amazon, Google, FB, etc) will keep re-interviewing you for the next few years. They know they make mistakes and want to try again under new circumstances. 2. If you actually are unqualified right now, you're very close to being qualified! If you've made it to onsite interviews then you're doing better than the majority of candidates. Most people are rejected at or before the phone screen. You're close, give yourself another year of improving your craft and interview again. Background: I worked at Amazon, and conducted interviews while there. I've conducted lots of interviews at a startup, I just interviewed at Google for the third time and finally got an offer (now 6 years after graduation). At all places, we optimized for false negatives.

vonmoltke 10y ago
> Any good company will have a high false negative rate, because it's more expensive to hire a bad candidate than reject a good one. As I usually do in these discussions, I would like to register that I think this is a load of crap. First, a good company will have a high F-measure. Optimizing your hiring process for precision and ignoring recall is just a sign of having a really deep, really talented applicant pool (or being deluded about the depth and talent of your pool). Second, when the costs of a bad hire come up lots of "cans" get thrown around without any grounding in reality. I view "optimiz[ing] for false negatives" the same way as I view security theater or overprotective parenting: attempts to stave off very low probability negative events that end up costing more than the events themselves would. Again, with a deep and talented pool this might not be the case, but if you are large enough to have that you are large enough that the damage a bad hire can cause is trivial to your business.
- myblake 10y ago
  I think it's probably much truer that big tech companies are optimizing for false negatives because they do have a really deep, talented applicant pool. Some start ups do but it's definitely more of a risk. I completely agree with you that in general you're better off optimizing the F-measure, but I'm not sure it's empirically true that lots of companies thought of as good do this.
  - vonmoltke 10y ago
    Thinking about this a little more, your final point may be right. It may be that companies become good by optimizing for F-measure, develop a talented and deep pool, then transition to optimizing for precision to manage the applicant load. Thus, most companies considered good have already transitioned while the ones who haven't aren't widely known.
- SomeCallMeTim 10y ago
  So you're actually arguing for companies to hire employees who will likely produce negative output (their code will be so bad as to take not only their own time, but other programmers' time to fix) just to ...what, not hurt the feelings of the bottom 80 percent of developers? Your analogy is poor: Hiring a bad programmer is an extremely high probability event, even in a broad pool of candidates as would apply to a high-end company like Amazon, not a low priority event. There have been studies dating back to the 70's that show a 10-20x skill difference between average and great developers. If you're going to pay roughly the same for a 1x as a 10x developer, wouldn't you want the 10x? So they tune their hiring process to skew that number as high as possible. OP above may be an exception who fell through the cracks, not really ragging on them in particular. I did know at least one person who applied to Amazon and who was rejected -- and rightfully so, in my opinion. His skills were sub-par, and the interview process detected that. Most others that I worked with while I was at Amazon were also above average; people I've encountered at other jobs have been mediocre by comparison, with the exception of a number of game developers I know.
  - vonmoltke 10y ago
    > So you're actually arguing for companies to hire employees who will likely produce negative output (their code will be so bad as to take not only their own time, but other programmers' time to fix) just to ...what, not hurt the feelings of the bottom 80 percent of developers? I did nothing of the sort. In fact, I am arguing that your hypothetical negative output employee is much rarer that some people think. > Your analogy is poor: Hiring a bad programmer is an extremely high probability event, even in a broad pool of candidates as would apply to a high-end company like Amazon, not a low priority event. I have seen no data to support this. My personal experience does not support this. > There have been studies dating back to the 70's that show a 10-20x skill difference between average and great developers. If you're going to pay roughly the same for a 1x as a 10x developer, wouldn't you want the 10x? So they tune their hiring process to skew that number as high as possible. Citation needed. I have certainly seen this bandied about the Internet quite a bit, but the best the citations only claim a 10x difference between great and terrible developers, and even those studies are likely flawed.[1] > I did know at least one person who applied to Amazon and who was rejected -- and rightfully so, in my opinion. His skills were sub-par, and the interview process detected that. Most others that I worked with while I was at Amazon were also above average; people I've encountered at other jobs have been mediocre by comparison, with the exception of a number of game developers I know. I'm not saying people shouldn't be rejected or that bad developers should have jobs at places like Amazon. I'm saying that optimizing for false negatives because you are paranoid of false positives is an anti-pattern and that there are better ways to deal with false positives. [1] http://blog.fogcreek.com/10x-programmer-and-other-myths-in-software-engineering-interview-with-laurent-bossavit/ http://blog.fogcreek.com/10x-programmer-and-other-myths-in-s...
- pyb 10y ago
  In my experience, there is a large hidden cost in rejecting good candidates. I've seen first hand companies go under and projects fail, because too few employees really had the experience or ability to tackle challenging projects.
aestetix 10y ago
"Any good company will have a high false negative rate, because it's more expensive to hire a bad candidate than reject a good one." Isn't part of good hiring practices to determine who is a good candidate? And beyond that, if the candidate is "bad", doesn't the company have the ability to work with the now employee to improve, and if things don't work out, to part ways? It seems like sending someone through endless loops of "maybe" interviews is a waste of both parties time.
akhilcacharya 10y ago
>That's why the big ones (Amazon, Google, FB, etc) will keep re-interviewing you for the next few years This applies to interns too, correct?

by buss 10y ago

vonmoltke 10y ago

> Any good company will have a high false negative rate, because it's more expensive to hire a bad candidate than reject a good one. As I usually do in these discussions, I would like to register that I think this is a load of crap. First, a good company will have a high F-measure. Optimizing your hiring process for precision and ignoring recall is just a sign of having a really deep, really talented applicant pool (or being deluded about the depth and talent of your pool). Second, when the costs of a bad hire come up lots of "cans" get thrown around without any grounding in reality. I view "optimiz[ing] for false negatives" the same way as I view security theater or overprotective parenting: attempts to stave off very low probability negative events that end up costing more than the events themselves would. Again, with a deep and talented pool this might not be the case, but if you are large enough to have that you are large enough that the damage a bad hire can cause is trivial to your business.

myblake 10y ago
I think it's probably much truer that big tech companies are optimizing for false negatives because they do have a really deep, talented applicant pool. Some start ups do but it's definitely more of a risk. I completely agree with you that in general you're better off optimizing the F-measure, but I'm not sure it's empirically true that lots of companies thought of as good do this.
- vonmoltke 10y ago
  Thinking about this a little more, your final point may be right. It may be that companies become good by optimizing for F-measure, develop a talented and deep pool, then transition to optimizing for precision to manage the applicant load. Thus, most companies considered good have already transitioned while the ones who haven't aren't widely known.
SomeCallMeTim 10y ago
So you're actually arguing for companies to hire employees who will likely produce negative output (their code will be so bad as to take not only their own time, but other programmers' time to fix) just to ...what, not hurt the feelings of the bottom 80 percent of developers? Your analogy is poor: Hiring a bad programmer is an extremely high probability event, even in a broad pool of candidates as would apply to a high-end company like Amazon, not a low priority event. There have been studies dating back to the 70's that show a 10-20x skill difference between average and great developers. If you're going to pay roughly the same for a 1x as a 10x developer, wouldn't you want the 10x? So they tune their hiring process to skew that number as high as possible. OP above may be an exception who fell through the cracks, not really ragging on them in particular. I did know at least one person who applied to Amazon and who was rejected -- and rightfully so, in my opinion. His skills were sub-par, and the interview process detected that. Most others that I worked with while I was at Amazon were also above average; people I've encountered at other jobs have been mediocre by comparison, with the exception of a number of game developers I know.
- vonmoltke 10y ago
  > So you're actually arguing for companies to hire employees who will likely produce negative output (their code will be so bad as to take not only their own time, but other programmers' time to fix) just to ...what, not hurt the feelings of the bottom 80 percent of developers? I did nothing of the sort. In fact, I am arguing that your hypothetical negative output employee is much rarer that some people think. > Your analogy is poor: Hiring a bad programmer is an extremely high probability event, even in a broad pool of candidates as would apply to a high-end company like Amazon, not a low priority event. I have seen no data to support this. My personal experience does not support this. > There have been studies dating back to the 70's that show a 10-20x skill difference between average and great developers. If you're going to pay roughly the same for a 1x as a 10x developer, wouldn't you want the 10x? So they tune their hiring process to skew that number as high as possible. Citation needed. I have certainly seen this bandied about the Internet quite a bit, but the best the citations only claim a 10x difference between great and terrible developers, and even those studies are likely flawed.[1] > I did know at least one person who applied to Amazon and who was rejected -- and rightfully so, in my opinion. His skills were sub-par, and the interview process detected that. Most others that I worked with while I was at Amazon were also above average; people I've encountered at other jobs have been mediocre by comparison, with the exception of a number of game developers I know. I'm not saying people shouldn't be rejected or that bad developers should have jobs at places like Amazon. I'm saying that optimizing for false negatives because you are paranoid of false positives is an anti-pattern and that there are better ways to deal with false positives. [1] http://blog.fogcreek.com/10x-programmer-and-other-myths-in-software-engineering-interview-with-laurent-bossavit/ http://blog.fogcreek.com/10x-programmer-and-other-myths-in-s...
pyb 10y ago
In my experience, there is a large hidden cost in rejecting good candidates. I've seen first hand companies go under and projects fail, because too few employees really had the experience or ability to tackle challenging projects.

aestetix 10y ago

"Any good company will have a high false negative rate, because it's more expensive to hire a bad candidate than reject a good one." Isn't part of good hiring practices to determine who is a good candidate? And beyond that, if the candidate is "bad", doesn't the company have the ability to work with the now employee to improve, and if things don't work out, to part ways? It seems like sending someone through endless loops of "maybe" interviews is a waste of both parties time.

akhilcacharya 10y ago

>That's why the big ones (Amazon, Google, FB, etc) will keep re-interviewing you for the next few years This applies to interns too, correct?