5 ms·

The thing that companies fail to do is reconcile hiring decisions with performance reviews afterwards. They have a single hiring practice and they NEVER EVER ch

The thing that companies fail to do is reconcile hiring decisions with performance reviews afterwards. They have a single hiring practice and they NEVER EVER change it, regardless of the quality of the eventual hires. The missing feedback loop from performance review back to the hiring process means that the hiring process can't get fixed. Until companies start adding that process back in, hiring will never get better.

pkaye 6y ago
I believe Google does this reconciliation between hiring decision and performance reviews.
tmsh 6y ago
I think it's complicated by the fact that the better the company, they better they are at mentoring even "bad" hires. I.e., performance review isn't normalized across all companies. Google might think their hiring process is great because a lot of their hires do well in their performance review - but maybe that has more to do with the engineering culture and mentorship in the company, and less to do with their ability to pick future winners. Perhaps though if one looks at the performance review weighted to a review date as soon after the hiring, and sort of tail off how much weight is given over the coming years it'd make sense. I've just seen a lot of bad hires get better with time. And some good hires lose steam or become jaded and perform worse over time. Thus, as messed up as it is, one of the keenest signals is tenacity in the end. And ironically tenacity in preparation for interviews (that Leetcode grind) is somewhat correlated with performance reviews in the long run and perhaps correlated with utility to the company. That said, "decent" engineers in the long run that don't think outside the box but check their performance review boxes slowly bleed a company's ability to disrupt. I.e., performance reviews need to be expanded a bit to think about the health of the overall company and not just the employee performing their function. As a large tech organization, I'd prefer a complex distribution of performances over an even distribution any day - because value to tech companies is still influenced by more black swan event type of innovation. But then I have a high "beta" (variation) in terms of what I'm looking for. Companies with more of a utility emphasis perhaps are better served by a hiring process that is "reliable" with a lot of false negatives.
- ghostpepper 6y ago
  If someone becomes a net-contributor after some mentorship, were they really a bad hire?
  - jennyyang 6y ago
    I guess if they can take someone they would not have hired, and then mentored them to become a net-contributor, then the hiring process is broken. You can't really test this without hiring people you wouldn't normally hire based on your hiring process.
- jennyyang 6y ago
  A company's hiring practice should reflect the type of engineer they want to hire. If they want to hire engineers that can start contributing immediately, and they hire engineers that need mentoring, then the hiring process is broken. If they widen their expectations, and hire people that they believe they can mentor into net contributors, then their hiring practices should reflect this. There are many ways a company can make hiring more scientific, including A/B testing (ie. hiring people they normally wouldn't, have more data around how productive new hires are, etc) but I've never seen it over than "copy Google and we should be good."
vladTheInhaler 6y ago
I agree that in general, feedback loops are important. But you had better be absolutely sure that you're measuring the right thing, because once you put it in place, your hiring process will begin to optimize around it. So unless you can write down precisely what it means to be a good whatever, you probably shouldn't expect feedback to solve your hiring problems.
- tmsh 6y ago
  Yeah, I think this is a more concise version of the point I was trying to make. And that perhaps "performance reviews" should be one of a couple of signals as feedback to the hiring process. There's just so many variables in play (covid-19, etc.) one has to be super careful about recognizing the right signals / causation, etc. Ideally one needs experiments about which signals or feedback to use in determining if it's a good hire in addition to experiments about different hiring process changes.
necubi 6y ago
An issue with this is that it's inherently one-sided: you get to see the performance of the people you hire, but you have no idea what happens with the people you reject. Because of that, you inevitably get "weird" results like seeing that your best employees often did relatively poorly on interviews (Google did a study that produced exactly that result—their best employees often were those that failed 1 or 2 of their interviews). But this is just selection bias: these are candidates that got hired! Which just means that they were so spectacular in some other respect that they were able to get an offer /despite/ poor interview performance. It tells you nothing actionable about your interview process. Really you need to take an experimental approach where you randomly give offers to people who failed the interview, and see how they turn out.
morgante 6y ago
I guarantee that companies (at least large companies) do reconcile hiring decisions with performance review data. Google, at least, has entire teams of analysts focused on exactly this sort of analysis. Fundamentally, hiring is simply not an easy problem. There's a small cohort of candidates who can be reasonably confident will perform well in a job (mostly people who have performed a similar job at a competitor) that everyone competes over and a marge larger group of candidates whose possible performance is very hard to judge.

pkaye 6y ago

I believe Google does this reconciliation between hiring decision and performance reviews.

tmsh 6y ago

I think it's complicated by the fact that the better the company, they better they are at mentoring even "bad" hires. I.e., performance review isn't normalized across all companies. Google might think their hiring process is great because a lot of their hires do well in their performance review - but maybe that has more to do with the engineering culture and mentorship in the company, and less to do with their ability to pick future winners. Perhaps though if one looks at the performance review weighted to a review date as soon after the hiring, and sort of tail off how much weight is given over the coming years it'd make sense. I've just seen a lot of bad hires get better with time. And some good hires lose steam or become jaded and perform worse over time. Thus, as messed up as it is, one of the keenest signals is tenacity in the end. And ironically tenacity in preparation for interviews (that Leetcode grind) is somewhat correlated with performance reviews in the long run and perhaps correlated with utility to the company. That said, "decent" engineers in the long run that don't think outside the box but check their performance review boxes slowly bleed a company's ability to disrupt. I.e., performance reviews need to be expanded a bit to think about the health of the overall company and not just the employee performing their function. As a large tech organization, I'd prefer a complex distribution of performances over an even distribution any day - because value to tech companies is still influenced by more black swan event type of innovation. But then I have a high "beta" (variation) in terms of what I'm looking for. Companies with more of a utility emphasis perhaps are better served by a hiring process that is "reliable" with a lot of false negatives.

ghostpepper 6y ago
If someone becomes a net-contributor after some mentorship, were they really a bad hire?
- jennyyang 6y ago
  I guess if they can take someone they would not have hired, and then mentored them to become a net-contributor, then the hiring process is broken. You can't really test this without hiring people you wouldn't normally hire based on your hiring process.
jennyyang 6y ago
A company's hiring practice should reflect the type of engineer they want to hire. If they want to hire engineers that can start contributing immediately, and they hire engineers that need mentoring, then the hiring process is broken. If they widen their expectations, and hire people that they believe they can mentor into net contributors, then their hiring practices should reflect this. There are many ways a company can make hiring more scientific, including A/B testing (ie. hiring people they normally wouldn't, have more data around how productive new hires are, etc) but I've never seen it over than "copy Google and we should be good."

vladTheInhaler 6y ago

I agree that in general, feedback loops are important. But you had better be absolutely sure that you're measuring the right thing, because once you put it in place, your hiring process will begin to optimize around it. So unless you can write down precisely what it means to be a good whatever, you probably shouldn't expect feedback to solve your hiring problems.

tmsh 6y ago
Yeah, I think this is a more concise version of the point I was trying to make. And that perhaps "performance reviews" should be one of a couple of signals as feedback to the hiring process. There's just so many variables in play (covid-19, etc.) one has to be super careful about recognizing the right signals / causation, etc. Ideally one needs experiments about which signals or feedback to use in determining if it's a good hire in addition to experiments about different hiring process changes.

necubi 6y ago

An issue with this is that it's inherently one-sided: you get to see the performance of the people you hire, but you have no idea what happens with the people you reject. Because of that, you inevitably get "weird" results like seeing that your best employees often did relatively poorly on interviews (Google did a study that produced exactly that result—their best employees often were those that failed 1 or 2 of their interviews). But this is just selection bias: these are candidates that got hired! Which just means that they were so spectacular in some other respect that they were able to get an offer /despite/ poor interview performance. It tells you nothing actionable about your interview process. Really you need to take an experimental approach where you randomly give offers to people who failed the interview, and see how they turn out.

morgante 6y ago

I guarantee that companies (at least large companies) do reconcile hiring decisions with performance review data. Google, at least, has entire teams of analysts focused on exactly this sort of analysis. Fundamentally, hiring is simply not an easy problem. There's a small cohort of candidates who can be reasonably confident will perform well in a job (mostly people who have performed a similar job at a competitor) that everyone competes over and a marge larger group of candidates whose possible performance is very hard to judge.