4 ms·

Please provide proof that a coding interview makes a statistically significant effect on predicting success in a position, all other things being equal.

SamReidHughes 4y ago
Compared to what alternative?
andreygrehov 4y ago
If I'm not mistaken, Google has a paper on this, in which there is a statistical confirmation of coding interviews being a predictor of success.
- nowherebeen 4y ago
  3 problems I can see with this a) Its like saying a Tobacco company has a paper saying smoking is healthy b) Google measures things that matter to them. The operate on a scale that 99% of companies aren't. Their predictor of success only applies to companies of their scale. c) What is their definition of success? And how does it apply to smaller companies less than 1000 employees?
  - Jensson 4y ago
    Google isn't selling its interview process. The one who is hurt if it doesn't work is Google itself, they have no reason to lie to themselves here. And I really doubt that someone tricked Larry Page into adopting this with faulty statistics, he cared so much about hiring that he personally reviewed every single hire until the company grew to over ten thousand employees. The hiring process changed many times over the years, and what Google runs now is what they ended up with.
    - nowherebeen 4y ago
      Reread a) and b) from another perspective because never did I say they were lying to themselves.
      - Jensson 4y ago
        I agree with b) and c). A is a bit forced though as tobacco companies faked studies to show that what they sell is healthy even though they knew it was horribly bad for you, I don't see how Google is anything like that with respect to hiring processes. If they talked about ads or search then the likeness would make sense, but I don't see it for hiring. But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. They can always use more developers to help optimize the profits of those algorithms or reduce runtime costs, the ROI on that is extremely high so that is what their hiring process is optimized for. Some of their other projects might have done better with a different process, but they don't have much weight compared to the revenue generating projects.
        nowherebeen 4y ago
        > But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. One of the Marques Brownlee videos mentions that when he met Eric Schmidt, Schmidt mentions that every problem Google have were scaling. If you look at it from his perspective, it would make a lot of sense why algo is so important for Google interviews. He applied it not only in tech, but with hiring as well.
        lapcat 4y ago
        "CEO Eric Eric Schmidt stood out because he 'was the only candidate who had been to Burning Man.'" https://en.wikipedia.org/wiki/Eric_Schmidt#cite_note-31 https://en.wikipedia.org/wiki/Eric_Schmidt#cite_note-31 Funny how the "objective tests" that leaders apply to grunts never get applied to the leaders themselves.
        4y ago
  - rvz 4y ago
    Precisely. Especially with (b) and (c) What works for Google does not mean it will also work for the 99% of companies that are NOT Google or any other FAANMG company. This only shows that we have an incessant coding interview cargo-cult around how FAANMG companies interview candidates and several startups taking that to the extreme and thinking they are working on complex Google scale problems. To these interviewers at these startups still under this delusion and for the 900th time as many have previously said for years as a wake up call: You are NOT Google.
    - nowherebeen 4y ago
      A wake up call is coming to theses startups because the money has stopped flowing. I suspect a lot of these startups replicating Google in both interview style and architecture are going together fail fast. Read up on HN post on Fast. Someone was commenting one how they failed the interview because the HM wanted someone that wasn't a cowboy. They wanted someone that build microservices like Google. Being a cowboy is what you need to find PM fit. Focusing on microservices when you don't even have enough customers is a huge waste of money and detrimental to your company because of operational complexity. Any changes that should take 5 mins, now takes 3 hours. Fast had so little customers that an engineer suggested that they run the system under a single AWS monolith and was quickly shot down.
- lordnacho 4y ago
  Did they hire people who failed their interviews in order to see how well they did?
  - higeorge13 4y ago
    That would be an amazing ab testing for hiring. And they are among the 100 companies who could easily do it.
sokoloff 4y ago
How could you reasonably prove this? “All other things being equal” seems harder to evaluate than a coding interview. Drive, experience, intelligence, communication skills, teamwork, emotional intelligence, grit, curiosity, physical health, mental stamina, family/home situation, and dozens of other factors play into job performance, many of which you can’t even ask about to generate your test dataset. Many of these traits are what you’re trying to measure the effects of in the interview.
manuelabeledo 4y ago
My experience is obviously just anecdotal, but I can say that in the severely limited candidate pool I worked with in the past decade, there has been no correlation between their proficiency at code interviews and their performance at their day to day job. I’m also not familiar with other fields, but are doctors or lawyers asked to do anything analogous to code assessments during interviews?

SamReidHughes 4y ago

Compared to what alternative?

andreygrehov 4y ago

If I'm not mistaken, Google has a paper on this, in which there is a statistical confirmation of coding interviews being a predictor of success.

nowherebeen 4y ago
3 problems I can see with this a) Its like saying a Tobacco company has a paper saying smoking is healthy b) Google measures things that matter to them. The operate on a scale that 99% of companies aren't. Their predictor of success only applies to companies of their scale. c) What is their definition of success? And how does it apply to smaller companies less than 1000 employees?
- Jensson 4y ago
  Google isn't selling its interview process. The one who is hurt if it doesn't work is Google itself, they have no reason to lie to themselves here. And I really doubt that someone tricked Larry Page into adopting this with faulty statistics, he cared so much about hiring that he personally reviewed every single hire until the company grew to over ten thousand employees. The hiring process changed many times over the years, and what Google runs now is what they ended up with.
  - nowherebeen 4y ago
    Reread a) and b) from another perspective because never did I say they were lying to themselves.
    - Jensson 4y ago
      I agree with b) and c). A is a bit forced though as tobacco companies faked studies to show that what they sell is healthy even though they knew it was horribly bad for you, I don't see how Google is anything like that with respect to hiring processes. If they talked about ads or search then the likeness would make sense, but I don't see it for hiring. But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. They can always use more developers to help optimize the profits of those algorithms or reduce runtime costs, the ROI on that is extremely high so that is what their hiring process is optimized for. Some of their other projects might have done better with a different process, but they don't have much weight compared to the revenue generating projects.
      - nowherebeen 4y ago
        > But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. One of the Marques Brownlee videos mentions that when he met Eric Schmidt, Schmidt mentions that every problem Google have were scaling. If you look at it from his perspective, it would make a lot of sense why algo is so important for Google interviews. He applied it not only in tech, but with hiring as well.
        lapcat 4y ago
        "CEO Eric Eric Schmidt stood out because he 'was the only candidate who had been to Burning Man.'" https://en.wikipedia.org/wiki/Eric_Schmidt#cite_note-31 https://en.wikipedia.org/wiki/Eric_Schmidt#cite_note-31 Funny how the "objective tests" that leaders apply to grunts never get applied to the leaders themselves.
        4y ago
- rvz 4y ago
  Precisely. Especially with (b) and (c) What works for Google does not mean it will also work for the 99% of companies that are NOT Google or any other FAANMG company. This only shows that we have an incessant coding interview cargo-cult around how FAANMG companies interview candidates and several startups taking that to the extreme and thinking they are working on complex Google scale problems. To these interviewers at these startups still under this delusion and for the 900th time as many have previously said for years as a wake up call: You are NOT Google.
  - nowherebeen 4y ago
    A wake up call is coming to theses startups because the money has stopped flowing. I suspect a lot of these startups replicating Google in both interview style and architecture are going together fail fast. Read up on HN post on Fast. Someone was commenting one how they failed the interview because the HM wanted someone that wasn't a cowboy. They wanted someone that build microservices like Google. Being a cowboy is what you need to find PM fit. Focusing on microservices when you don't even have enough customers is a huge waste of money and detrimental to your company because of operational complexity. Any changes that should take 5 mins, now takes 3 hours. Fast had so little customers that an engineer suggested that they run the system under a single AWS monolith and was quickly shot down.
lordnacho 4y ago
Did they hire people who failed their interviews in order to see how well they did?
- higeorge13 4y ago
  That would be an amazing ab testing for hiring. And they are among the 100 companies who could easily do it.

sokoloff 4y ago

How could you reasonably prove this? “All other things being equal” seems harder to evaluate than a coding interview. Drive, experience, intelligence, communication skills, teamwork, emotional intelligence, grit, curiosity, physical health, mental stamina, family/home situation, and dozens of other factors play into job performance, many of which you can’t even ask about to generate your test dataset. Many of these traits are what you’re trying to measure the effects of in the interview.

manuelabeledo 4y ago

My experience is obviously just anecdotal, but I can say that in the severely limited candidate pool I worked with in the past decade, there has been no correlation between their proficiency at code interviews and their performance at their day to day job. I’m also not familiar with other fields, but are doctors or lawyers asked to do anything analogous to code assessments during interviews?