5 ms·

I work for a big tech, and I have yet to see any evidence that leetcode-style interviews lead to better hires. If that data is being tracked anywhere, it's kept

I work for a big tech, and I have yet to see any evidence that leetcode-style interviews lead to better hires. If that data is being tracked anywhere, it's kept very secret. In fact the metric that recruitment uses for success is "inclined rate". It's easy to see how that is a lousy metric: I can just say yes to all candidates, but that doesn't mean they will do a good job.

tzhenghao 4y ago
I think it has to do with headcount requirements once you're Google scale too. Leetcode style interviews become yet another "process to avoid disaster at a big company". Disaster here means hiring a bad engineer and then having to fire them for performance reasons or whatever. Google's whole concept of hiring committees like it was graduate school also reinforces this. They probably do this to minimize nepotism at the hiring stage. All these processes become items on a hiring checklist, but to your point, it doesn't guarantee hiring a great engineer. Just one of those tragedies embedded alongside other big co bureaucracy processes.
potamic 4y ago
What is inclined rate?
- Root_Denied 4y ago
  "Inclined To Hire", meaning someone who either interviewed the person directly or reviewed their hiring profile and all the interview feedback is saying yes, we should hire this person. Basically the equivalent of the Roman emperor thumbs up/thumbs down except it's a small group and you need a certain number of thumbs up to move forward. The number of people you need Inclined to Hire differs from org to org that use this method.
Jensson 4y ago
> I work for a big tech, and I have yet to see any evidence that leetcode-style interviews lead to better hires. I worked for Google and at least back then everyone working there could read the studies showing that the interviews worked. The effect was very clear, with there being no upper bar where the interview score stopped mattering. If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. But I'm not sure why Google would lie about that, or why they would keep the interview format for so long if it didn't work. The old article talking about Google interviews talked about what they did before they arrived at this format, what they do today is the best interview according to their data.
- throwawausjd 4y ago
  So his position is weak but you’re not in the strongest position either. Basically, the same depth of evidence (maybe less) that saturated fat was causing all this heart disease, so let’s keep everyone on very low fat diets. I just remained unconvinced that we know anything at this point. Google is allowed to do what they’re going to do and to do their own studies and make their own decisions based on that data. However, that study is not a normal lock for the rest of the world and people SHOULD be questioning it, even with anecdotal evidence, at least till we have enough studies proving their point in different situations and companies.
  - ctvo 4y ago
    I agree, we don't have enough data. And a common mistake is extrapolating too much from limited data, but here's the most important data point and why we continue to use these practices: these companies are worth trillions of dollars and they hire this way. You want the industry to change its practices? Find an equally successful (impact, market cap, innovation) company that hires differently in software.
- jen20 4y ago
  I haven’t seen this mythical closed data, but I have used Google products, so I’m inclined not to believe that whatever they do leads to acceptable results.
- angarg12 4y ago
  That's a very judgmental reply that also misrepresent my comment. I don't work at Google, so I cannot speak for them. My company does have studies and experiments around interviewing, but again, the metric they use for success is "inclined rate" e.g. doing ABC leads to x% increase in inclined rate. I haven't seen any study linking interview outcomes with on the job performance e.g. yearly performance rating or promotions. In fact many people have requested this data over the years. > If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. This comment seems quite rude, and again, it misrepresents my position. I didn't say LC style interviews don't work, I'm saying that I don't know, because I have seen no data to prove that they work better or worse than other alternatives. Also the landscape has change, and what might have worked 5 or 10 years might not anymore. Tech interviewing has "professionalized" in a Goodhart's Law-esque fashion. People have become very dedicated to game the process, which might affect the quality of the outcomes. But just to be clear again, I want to see evidence one way or another.

tzhenghao 4y ago

I think it has to do with headcount requirements once you're Google scale too. Leetcode style interviews become yet another "process to avoid disaster at a big company". Disaster here means hiring a bad engineer and then having to fire them for performance reasons or whatever. Google's whole concept of hiring committees like it was graduate school also reinforces this. They probably do this to minimize nepotism at the hiring stage. All these processes become items on a hiring checklist, but to your point, it doesn't guarantee hiring a great engineer. Just one of those tragedies embedded alongside other big co bureaucracy processes.

potamic 4y ago

What is inclined rate?

Root_Denied 4y ago
"Inclined To Hire", meaning someone who either interviewed the person directly or reviewed their hiring profile and all the interview feedback is saying yes, we should hire this person. Basically the equivalent of the Roman emperor thumbs up/thumbs down except it's a small group and you need a certain number of thumbs up to move forward. The number of people you need Inclined to Hire differs from org to org that use this method.

Jensson 4y ago

> I work for a big tech, and I have yet to see any evidence that leetcode-style interviews lead to better hires. I worked for Google and at least back then everyone working there could read the studies showing that the interviews worked. The effect was very clear, with there being no upper bar where the interview score stopped mattering. If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. But I'm not sure why Google would lie about that, or why they would keep the interview format for so long if it didn't work. The old article talking about Google interviews talked about what they did before they arrived at this format, what they do today is the best interview according to their data.

throwawausjd 4y ago
So his position is weak but you’re not in the strongest position either. Basically, the same depth of evidence (maybe less) that saturated fat was causing all this heart disease, so let’s keep everyone on very low fat diets. I just remained unconvinced that we know anything at this point. Google is allowed to do what they’re going to do and to do their own studies and make their own decisions based on that data. However, that study is not a normal lock for the rest of the world and people SHOULD be questioning it, even with anecdotal evidence, at least till we have enough studies proving their point in different situations and companies.
- ctvo 4y ago
  I agree, we don't have enough data. And a common mistake is extrapolating too much from limited data, but here's the most important data point and why we continue to use these practices: these companies are worth trillions of dollars and they hire this way. You want the industry to change its practices? Find an equally successful (impact, market cap, innovation) company that hires differently in software.
jen20 4y ago
I haven’t seen this mythical closed data, but I have used Google products, so I’m inclined not to believe that whatever they do leads to acceptable results.
angarg12 4y ago
That's a very judgmental reply that also misrepresent my comment. I don't work at Google, so I cannot speak for them. My company does have studies and experiments around interviewing, but again, the metric they use for success is "inclined rate" e.g. doing ABC leads to x% increase in inclined rate. I haven't seen any study linking interview outcomes with on the job performance e.g. yearly performance rating or promotions. In fact many people have requested this data over the years. > If you haven't seen any data then you just have a set of anecdotes, so your position is very weak. I have seen the data, but a person like you who is convinced it work wont believe me of course. This comment seems quite rude, and again, it misrepresents my position. I didn't say LC style interviews don't work, I'm saying that I don't know, because I have seen no data to prove that they work better or worse than other alternatives. Also the landscape has change, and what might have worked 5 or 10 years might not anymore. Tech interviewing has "professionalized" in a Goodhart's Law-esque fashion. People have become very dedicated to game the process, which might affect the quality of the outcomes. But just to be clear again, I want to see evidence one way or another.