5 ms·

[author here] Hi all - I hope you find this useful! If you have questions or want to know more I’m hosting a live stream Q&A today on Twitter at 1 PM Pacific.

[author here] Hi all - I hope you find this useful! If you have questions or want to know more I’m hosting a live stream Q&A today on Twitter at 1 PM Pacific. Tune in and bring your interviewing and hiring questions! I'm @startupandrew on Twitter.

doctorpangloss 7y ago
Was there ever a time the testing and quizzing was wrong? How did you know? For example, “Later that candidate did X, which would have been great for Firebase.” How often did you take subordinates that weren’t fresh college graduates or former/current FAANG engineers?
- mayop100 7y ago
  Yes, we definitely had false negatives. It's always hard to be sure if you made a mistake though -- someone might be super successful at another company but not work out well at yours. I'm sure we made mistakes, though I don't have stats on this. We definitely hired non-new-grad non-FAANG engineers. In fact, new grads and FAANG alums were a minority for us.
  - vonmoltke 7y ago
    > someone might be super successful at another company but not work out well at yours I don't actually buy this. Environment makes a difference, sure, but if someone goes off and succeeds technically[1] at a another company in a big way they would have succeeded at yours as well. If that person can't succeed with you, then there is something fundamentally wrong with your organization. My experience, though, is that such organizations have much lower hiring bars. [1] I say "technically" because it's quite possible that the person is not very good technically, but landed at a place where that didn't matter and some combination of non-technical skills led to their success.
jakebasile 7y ago
Your claim that a take-home test is "lower stress" is undercut by the fact that you time your interviewees on their homework. Why do you think it is important to gauge how fast a person might be able to solve your toy problem, and why do you expect your candidates to dedicate 6-12 hours (unpaid) of continuous time to your company before you've even met them in person in most cases? Did you consider that this might inherently discriminate against those with significant time constraints in addition to their current job, biasing your results to new grads and people without hobbies and/or families?
- mayop100 7y ago
  The alternative to a take-home test is on-site interviews with white board problems. These also take (unpaid) time and are timed (you have until the end of the interview). Doing this part of the interview at-home rather than in-person gives people more schedule flexibility, not less. The total time spent is higher than most interview processes I think, which can be a struggle for some folks. But I do think it's the right tradeoff. Regarding why time taken is an important data point -- we found a strong correlation between people who were able to get good solutions working quickly and people who were able to effectively defend their solutions during the "GoldMine review" in their on-site interview.
  - jakebasile 7y ago
    > The alternative to a take-home test is on-site interviews with white board problems. That's not the only alternative though. Why not use a more humane interview process involving conversation about previous work, ideas about the future, strengths and weaknesses? > The total time spent is higher than most interview processes I think, which can be a struggle for some folks. But I do think it's the right tradeoff. I don't agree but I'm glad you thought about it. I personally think making your interview process more inclusive allows you to get better candidates, as opposed to structuring the process in such a way to exclude those that have different priorities and life choices. > Regarding why time taken is an important data point -- we found a strong correlation between people who were able to get good solutions working quickly and people who were able to effectively defend their solutions during the "GoldMine review" in their on-site interview. Correlation doesn't equal causation though. I also want to point out that if you are expecting someone to "defend" themselves, that means you are on the offensive. Interviews don't need to be adversarial. Edit: I do want to put in one good thing I got from your article to offset my rather negative responses. You mention that even if you reject a candidate you gave them the full code review and feedback about why you rejected them. The culture of secrecy surrounding rejections is something that needs to die, and I'm glad that although I think your process as a whole is unfair, you took the step to change that toxic part of our industry's interview culture.

doctorpangloss 7y ago

Was there ever a time the testing and quizzing was wrong? How did you know? For example, “Later that candidate did X, which would have been great for Firebase.” How often did you take subordinates that weren’t fresh college graduates or former/current FAANG engineers?

mayop100 7y ago
Yes, we definitely had false negatives. It's always hard to be sure if you made a mistake though -- someone might be super successful at another company but not work out well at yours. I'm sure we made mistakes, though I don't have stats on this. We definitely hired non-new-grad non-FAANG engineers. In fact, new grads and FAANG alums were a minority for us.
- vonmoltke 7y ago
  > someone might be super successful at another company but not work out well at yours I don't actually buy this. Environment makes a difference, sure, but if someone goes off and succeeds technically[1] at a another company in a big way they would have succeeded at yours as well. If that person can't succeed with you, then there is something fundamentally wrong with your organization. My experience, though, is that such organizations have much lower hiring bars. [1] I say "technically" because it's quite possible that the person is not very good technically, but landed at a place where that didn't matter and some combination of non-technical skills led to their success.

jakebasile 7y ago

Your claim that a take-home test is "lower stress" is undercut by the fact that you time your interviewees on their homework. Why do you think it is important to gauge how fast a person might be able to solve your toy problem, and why do you expect your candidates to dedicate 6-12 hours (unpaid) of continuous time to your company before you've even met them in person in most cases? Did you consider that this might inherently discriminate against those with significant time constraints in addition to their current job, biasing your results to new grads and people without hobbies and/or families?

mayop100 7y ago
The alternative to a take-home test is on-site interviews with white board problems. These also take (unpaid) time and are timed (you have until the end of the interview). Doing this part of the interview at-home rather than in-person gives people more schedule flexibility, not less. The total time spent is higher than most interview processes I think, which can be a struggle for some folks. But I do think it's the right tradeoff. Regarding why time taken is an important data point -- we found a strong correlation between people who were able to get good solutions working quickly and people who were able to effectively defend their solutions during the "GoldMine review" in their on-site interview.
- jakebasile 7y ago
  > The alternative to a take-home test is on-site interviews with white board problems. That's not the only alternative though. Why not use a more humane interview process involving conversation about previous work, ideas about the future, strengths and weaknesses? > The total time spent is higher than most interview processes I think, which can be a struggle for some folks. But I do think it's the right tradeoff. I don't agree but I'm glad you thought about it. I personally think making your interview process more inclusive allows you to get better candidates, as opposed to structuring the process in such a way to exclude those that have different priorities and life choices. > Regarding why time taken is an important data point -- we found a strong correlation between people who were able to get good solutions working quickly and people who were able to effectively defend their solutions during the "GoldMine review" in their on-site interview. Correlation doesn't equal causation though. I also want to point out that if you are expecting someone to "defend" themselves, that means you are on the offensive. Interviews don't need to be adversarial. Edit: I do want to put in one good thing I got from your article to offset my rather negative responses. You mention that even if you reject a candidate you gave them the full code review and feedback about why you rejected them. The culture of secrecy surrounding rejections is something that needs to die, and I'm glad that although I think your process as a whole is unfair, you took the step to change that toxic part of our industry's interview culture.