3 ms·

It's very easy to achieve if your process is designed to be correlated with success on the job. Most aren't though, they're designed as a contrived game that's

It's very easy to achieve if your process is designed to be correlated with success on the job. Most aren't though, they're designed as a contrived game that's tangentially related to performance on the job. Which works decent enough if you fire the bottom 20%, but is totally avoidable Having an authoritative body to make the final hire/no hire decision is important too. Needs to be tightly controlled and consistent. Letting random HMs make the decision will inevitably lead to decline in quality. If somebody who's good at their job can't do well on your interview without studying, it's a sign your process is not assessing what matters. A good process will select for people who are good, despite whether they study beforehand or not. I've never had high conviction in a hire and have them underperform on the job. If you do, you need to reassess what criteria you're using to make hiring decisions. (Never recently, in the early days, yes)

mlyle 4y ago
I don't like the excessive puzzling and contrived interview processes with excessive whiteboard coding, grilling, etc. I still think outperforming 1 in 10 is very hard. Especially since I've had to hire more people than I've had "high conviction" in. And beyond that, you just can't assess work ethic, diligence, attention to detail, etc, from an interview. You can get some secondary indicators that are correlated with that on the job, but imperfectly so.
- adam_arthur 4y ago
  If you're hiring people you don't have high conviction in, then of course you'll get relatively worse outcomes. I would question why. I do think companies need to set a target aptitude percentile they're trying to hire, as it has to be somewhat aligned with comp percentile to have success. You can't try to hire 99th percentile engineers with 50th percentile pay and expect to land any hires But if you set a lower target skill percentile to match lower comp, then you're actively choosing to lower the bar, so there should also be a lower bar for considering somebody a poor hire. At least in our case we were able to get by targeting ~80th percentile engineers at like 60th percentile comp. Mostly through hiring people that didn't know their worth in the market, but also being a good place to work, with low attrition (non comp factors)
  - mlyle 4y ago
    It seems like you've been hiring in domains that are a bit less technically demanding than the ones I was in. Even with a whole lot spent on recruiting and careful resume screens, about 5% of interviewees were people I was 95% confident could do distributed systems work; another 10% of interviewees were people that I was 90% confident could do the work. Turning the comp knob doesn't affect these numbers that much. So the acceptance criteria to get a reasonable hire rate and not spend all of your time interviewing becomes that 90% bar and being able to imagine working with that person without any obvious toxic or crazy behavior. In turn, that becomes about 15-20% not working out after hire. Setting the bar to 95% and really sure of cultural fit would be hiring at 1/3rd to 1/4th the pace, and you'd still have 7-8% not working out, which isn't that optimum of an operating point. I'd also have missed the absolute best performer I ever hired.
    - adam_arthur 4y ago
      Sure the domain of the work matters a lot. But the concept of skill distribution applies to all domains. You can hire a 10x engineer in a basic web dev role the same as you can hire a 10x in a distributed systems one. It's possible distributed systems work is inherently more difficult to assess, but I'm not sure. It seems the process just needs to be different to assess that skillset. If your best hire is somebody you had less confidence in, then it's a sign your confidence function is not fully tuned to what's causal to success. But you're right that you can't know everything after an interview. Work ethic cannot be measured directly as you stated, but usually it can be inferred with a reasonable level of confidence through behavioral cues of the candidate. A perfect correlation between hire confidence and performance on the job implies you have a perfect hiring process. Obviously impossible in practice, but you can get a lot closer than the average hiring process, or FAANG. Obviously to iteratively improve your hiring process, it's important to view it this way. Unfortunately most confuse and forget about what the relationship between hiring and on the job performance should be, and see hiring as a separate siloed game and view signals in the context of that game rather than the job. Finally, it's pretty bad for company culture and morale to have a high fire rate. It's worth going through more interviews to limit the bad hires. But obviously the tradeoff needs to be quantified and deliberately decided on. If your average tenure is < 2 years, there's something wrong

mlyle 4y ago

I don't like the excessive puzzling and contrived interview processes with excessive whiteboard coding, grilling, etc. I still think outperforming 1 in 10 is very hard. Especially since I've had to hire more people than I've had "high conviction" in. And beyond that, you just can't assess work ethic, diligence, attention to detail, etc, from an interview. You can get some secondary indicators that are correlated with that on the job, but imperfectly so.

adam_arthur 4y ago
If you're hiring people you don't have high conviction in, then of course you'll get relatively worse outcomes. I would question why. I do think companies need to set a target aptitude percentile they're trying to hire, as it has to be somewhat aligned with comp percentile to have success. You can't try to hire 99th percentile engineers with 50th percentile pay and expect to land any hires But if you set a lower target skill percentile to match lower comp, then you're actively choosing to lower the bar, so there should also be a lower bar for considering somebody a poor hire. At least in our case we were able to get by targeting ~80th percentile engineers at like 60th percentile comp. Mostly through hiring people that didn't know their worth in the market, but also being a good place to work, with low attrition (non comp factors)
- mlyle 4y ago
  It seems like you've been hiring in domains that are a bit less technically demanding than the ones I was in. Even with a whole lot spent on recruiting and careful resume screens, about 5% of interviewees were people I was 95% confident could do distributed systems work; another 10% of interviewees were people that I was 90% confident could do the work. Turning the comp knob doesn't affect these numbers that much. So the acceptance criteria to get a reasonable hire rate and not spend all of your time interviewing becomes that 90% bar and being able to imagine working with that person without any obvious toxic or crazy behavior. In turn, that becomes about 15-20% not working out after hire. Setting the bar to 95% and really sure of cultural fit would be hiring at 1/3rd to 1/4th the pace, and you'd still have 7-8% not working out, which isn't that optimum of an operating point. I'd also have missed the absolute best performer I ever hired.
  - adam_arthur 4y ago
    Sure the domain of the work matters a lot. But the concept of skill distribution applies to all domains. You can hire a 10x engineer in a basic web dev role the same as you can hire a 10x in a distributed systems one. It's possible distributed systems work is inherently more difficult to assess, but I'm not sure. It seems the process just needs to be different to assess that skillset. If your best hire is somebody you had less confidence in, then it's a sign your confidence function is not fully tuned to what's causal to success. But you're right that you can't know everything after an interview. Work ethic cannot be measured directly as you stated, but usually it can be inferred with a reasonable level of confidence through behavioral cues of the candidate. A perfect correlation between hire confidence and performance on the job implies you have a perfect hiring process. Obviously impossible in practice, but you can get a lot closer than the average hiring process, or FAANG. Obviously to iteratively improve your hiring process, it's important to view it this way. Unfortunately most confuse and forget about what the relationship between hiring and on the job performance should be, and see hiring as a separate siloed game and view signals in the context of that game rather than the job. Finally, it's pretty bad for company culture and morale to have a high fire rate. It's worth going through more interviews to limit the bad hires. But obviously the tradeoff needs to be quantified and deliberately decided on. If your average tenure is < 2 years, there's something wrong