8 ms·

Actually, it happens as long as your "stricter hiring practices" increase your false negative percentage by a lot more than they decrease your false positive pe

by fsk 12y ago

Actually, it happens as long as your "stricter hiring practices" increase your false negative percentage by a lot more than they decrease your false positive percentage. Try it out with some numbers. 10100 candidates, 100 are "good". Suppose you have 2% false positives and 1% false negatives. You hire 99 good candidates and 200 bad candidates. Suppose now you have 0.5% false positives and 90% false negatives. (You decreased your false positive rate by 4x but increased your false negative rate by 90x. This is typical for employers who look for every little excuse to reject someone.) You hire 10 good candidates and 50 bad candidates. Your "good hire" percentage went down, and you're churning through a lot more candidates to meet your hiring quota! So, "it is better to pass on a good candidate than hire a bad candidate" is FALSE if you wind up being too picky on passing on good candidates. Assuming you can identify losers and fire them after a year or two (with decent severance to be fair), you're actually better off hiring more leniently. It's also even worse when you realize that the candidate pool is more like: 10200 candidates, 100 are "good", 100 are "toxic", and the toxic people excel at pretending to be "good". Also, the rules for hiring are different for a famous employer and a no-name employer. Google and Facebook are going to have everyone competent applying. If you're a no-name startup, you'll be lucky to have 1 or 2 highly skilled candidates in your hiring pool.

fsk 12y ago
Also, what makes this mistake common is the feedback you get. When you make a false negative, you never find out that you passed on someone amazing. When you make a false positive, it's professional embarrassment for the boss when he's forced to admit he made a mistake and fire them. So the incentive for the boss is to minimize false positives, even at the expense of too many false negatives. The boss is looking out for his personal interests, and not what's best for the business.
- din-9 12y ago
  Reasons for not wanting to fire people also include: basic humanity.
  - fsk 12y ago
    But it's inhumane to your other employees to keep someone unqualified around.
frostmatthew 12y ago
> Try it out with some numbers. 10100 candidates, 100 are "good". What you're attempting to do works well for hypothetical drug testing[1] or terrorists but not for hiring developers (or anyone else). With the numbers you used you're proposing that less than 1% of all candidates are "good" - nobody would reasonably set the "good" threshold to include only the top 1% of developers. [1] https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing
- fsk 12y ago
  Even if you set the "good" percentage to 10%, too high a false positive rate will still ruin your results. Based on the people I've worked with over the years, I say that the actual skill distribution is: 5% toxic - These are the people who will ruin your business while deflecting blame to other people. 25% subtractors - These are the people who need more attention and help than the amount of work they get done. In the right environment, they can be useful. (Also, this is mostly independent of experience level. I know some really experienced people who were subtractors.) 60% average - These people are competent but not brilliant. These are solid performers. 9% above average - They can get 2x-5x the work done of someone average. 1% brilliant - These are the mythical 10x-100x programmers. These are the people who can write your MVP by themselves in 1-3 months and it'll be amazing. You first have to decide if you're targeting brilliant, above average, or average. For most businesses, average is good enough. If you incorrectly weed out the rare brilliant person, you might wind up instead with someone average, above average, or (even worse) toxic. Actually, when my employer was interviewing, I was surprised that the candidates were so strong. There was one brilliant guy and one above-average guy (My coworkers didn't like them; they failed the technical screening, which makes me distrust technical screening even more now). They wound up hiring one of the weakest candidates, a subtractor, and having worked with him for a couple of months my analysis of him hasn't changed.
  - frostmatthew 12y ago
    > 9% above average There is no reasonable definition of average that would only allow for 9% above that (or 10% including the 1% you marked as brilliant). Average is usually considered as either the 50th percentile (in which case you would have ~50% above this) or some middle range (e.g. 25th - 75th percentile). Since you said 60% are average we'll consider an appropriate range as average, the 20th - 80th percentile. That leaves you with 20% of applicants below average and 20% above. Your math falls apart real quick when we're dealing with distributions like 20%/60%/20% instead of 99.5%/0.5%. [As an aside, the toxics and brilliants are outliers, they should be fairly obvious to a competent interviewer (and as someone who previously spent a decade in an industry where nobody conducts interviews without adequate training I'll be the first to say most interviewers in our industry are not competent)].
skybrian 12y ago
This is reasoning from something that's harder to estimate (how good is the hiring pool and your interview process) to something you know more about (how good are the company's employees). It seems like you should be working backwards instead? For example, if you assume that 90% of your employees are "good" and 1% are "toxic", what does that tell you about the candidate pool and/or your interview process?
- fsk 12y ago
  It's my crude estimate based on the places I've worked over the years, and the people I've seen come in when my employers were interviewing. If I was the boss and had a "toxic" employee, I'd just dump them rather than waiting. I've been forced to work with toxic people because I'm not the boss, and I've noticed that toxic people are really good at pretending to be brilliant. Over the years, I've also worked with a couple of people who singlehandedly wrote all of the employer's key software. I also worked with several people who wrote a garbage system but conned everyone else into thinking it was brilliant. If 90% of candidates are "good", then why waste time with a detailed technical screening at all? Just interview a couple and pick the ones you like the best.

by fsk 12y ago

fsk 12y ago

Also, what makes this mistake common is the feedback you get. When you make a false negative, you never find out that you passed on someone amazing. When you make a false positive, it's professional embarrassment for the boss when he's forced to admit he made a mistake and fire them. So the incentive for the boss is to minimize false positives, even at the expense of too many false negatives. The boss is looking out for his personal interests, and not what's best for the business.

din-9 12y ago
Reasons for not wanting to fire people also include: basic humanity.
- fsk 12y ago
  But it's inhumane to your other employees to keep someone unqualified around.

frostmatthew 12y ago

> Try it out with some numbers. 10100 candidates, 100 are "good". What you're attempting to do works well for hypothetical drug testing[1] or terrorists but not for hiring developers (or anyone else). With the numbers you used you're proposing that less than 1% of all candidates are "good" - nobody would reasonably set the "good" threshold to include only the top 1% of developers. [1] https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing

fsk 12y ago
Even if you set the "good" percentage to 10%, too high a false positive rate will still ruin your results. Based on the people I've worked with over the years, I say that the actual skill distribution is: 5% toxic - These are the people who will ruin your business while deflecting blame to other people. 25% subtractors - These are the people who need more attention and help than the amount of work they get done. In the right environment, they can be useful. (Also, this is mostly independent of experience level. I know some really experienced people who were subtractors.) 60% average - These people are competent but not brilliant. These are solid performers. 9% above average - They can get 2x-5x the work done of someone average. 1% brilliant - These are the mythical 10x-100x programmers. These are the people who can write your MVP by themselves in 1-3 months and it'll be amazing. You first have to decide if you're targeting brilliant, above average, or average. For most businesses, average is good enough. If you incorrectly weed out the rare brilliant person, you might wind up instead with someone average, above average, or (even worse) toxic. Actually, when my employer was interviewing, I was surprised that the candidates were so strong. There was one brilliant guy and one above-average guy (My coworkers didn't like them; they failed the technical screening, which makes me distrust technical screening even more now). They wound up hiring one of the weakest candidates, a subtractor, and having worked with him for a couple of months my analysis of him hasn't changed.
- frostmatthew 12y ago
  > 9% above average There is no reasonable definition of average that would only allow for 9% above that (or 10% including the 1% you marked as brilliant). Average is usually considered as either the 50th percentile (in which case you would have ~50% above this) or some middle range (e.g. 25th - 75th percentile). Since you said 60% are average we'll consider an appropriate range as average, the 20th - 80th percentile. That leaves you with 20% of applicants below average and 20% above. Your math falls apart real quick when we're dealing with distributions like 20%/60%/20% instead of 99.5%/0.5%. [As an aside, the toxics and brilliants are outliers, they should be fairly obvious to a competent interviewer (and as someone who previously spent a decade in an industry where nobody conducts interviews without adequate training I'll be the first to say most interviewers in our industry are not competent)].

skybrian 12y ago

This is reasoning from something that's harder to estimate (how good is the hiring pool and your interview process) to something you know more about (how good are the company's employees). It seems like you should be working backwards instead? For example, if you assume that 90% of your employees are "good" and 1% are "toxic", what does that tell you about the candidate pool and/or your interview process?

fsk 12y ago
It's my crude estimate based on the places I've worked over the years, and the people I've seen come in when my employers were interviewing. If I was the boss and had a "toxic" employee, I'd just dump them rather than waiting. I've been forced to work with toxic people because I'm not the boss, and I've noticed that toxic people are really good at pretending to be brilliant. Over the years, I've also worked with a couple of people who singlehandedly wrote all of the employer's key software. I also worked with several people who wrote a garbage system but conned everyone else into thinking it was brilliant. If 90% of candidates are "good", then why waste time with a detailed technical screening at all? Just interview a couple and pick the ones you like the best.