4 ms·

I appreciate your honest feedback here! I’m currently the hiring manager for this position, and I agree that this doesn’t sound like a good experience. I disagr

I appreciate your honest feedback here! I’m currently the hiring manager for this position, and I agree that this doesn’t sound like a good experience. I disagree that a take-home assignment is inherently a bad idea - I actually have found a lot of value in seeing the work people produce there and feel it’s the part of the interview that’s closest to what the team actually does day-to-day - but I also don’t want candidates to feel like their efforts are being graded in a way that’s unfair or lacks transparency. Because of that, in the time since you interviewed, we’ve updated the rubric to be less subjective and better set up to reward the diverse ways someone could show data science skill, updated the text of the assignment to more clearly indicate what we’ll be evaluating, and made it standard practice to share with candidates right away where they did well and where they lost points. If I'm reading your concerns correctly, I think those changes are likely to address many of them.

mlthoughts2018 8y ago
Sounds like you’re doing a lot of good work on the process since the time of my experience. As long as the full set of evaluation criteria and expectations are made clear, and they sincerely fit into the time limit given, a take-home test can be perfectly fine. The trouble is two-fold - the test creators usually drastically underestimate how long the data canonicalization / normalization process takes just to get up & running with a toy data set. This stuff plus cursory data exploration always takes an hour or two — so code tests asking you to just hack together a model plus a write-up in a couple of hours are just not realistic. - tests are often subverted for political reasons. Someone doesn’t want to hire a candidate who is more skilled than they are; someone wants to recommend their college friend for the role; someone just prefers to work with extraverts but the candidate comes across as an introvert on the phone — whatever it is. Then arbitrary and secondary aspects of a test solution get amplified for debate to steer the decision in a political direction. Nobody wants to believe their teammates would do this, and many times people do it without realizing. But I’ve seen my own team doing it in the past during interview evaluations, and it’s particularly bad when debating over how to assess a take-home test submission. As a candidate I experienced it a few times too, notably with Asana as mentioned before. The other sad part of that Asana interview was that nobody in any part of the Asana pipeline ever asked me about my work experience. Every interview was tech questions, programming questions, and the take-home test. Not a single question about my work history. In the test feedback, the reviewer suggested I might not work well in a product-oriented environment — yet nobody had asked about my work experience in a product-oriented environment at an education technology company. That’s just one example, but there were others. It saddens me a lot that the tech industry is like this. It’s so myopic to believe I could assess someone’s subjective disposition towards being results-oriented or customer-focused or product-minded based on some random 3-hour test, and never even so much as ask them about their actual work history, where they actually solved problems and delivered products. It doesn’t feel good to be told, “based on this ambiguous code test, you’re probably not good at thinking about product development,” when you’ve got plenty of product-development items on your resume you’d be happy to share yet are never even asked about.
- erinakinci 8y ago
  I continue to agree that it doesn't sound like this was a good experience. I think you'd find that our data science interview process is actually substantially different today and changes many of the things that felt frustrating to you when you went through it a few years ago, including an increased focus on discussing past experience. Thank you again for your candid feedback.

mlthoughts2018 8y ago

Sounds like you’re doing a lot of good work on the process since the time of my experience. As long as the full set of evaluation criteria and expectations are made clear, and they sincerely fit into the time limit given, a take-home test can be perfectly fine. The trouble is two-fold - the test creators usually drastically underestimate how long the data canonicalization / normalization process takes just to get up & running with a toy data set. This stuff plus cursory data exploration always takes an hour or two — so code tests asking you to just hack together a model plus a write-up in a couple of hours are just not realistic. - tests are often subverted for political reasons. Someone doesn’t want to hire a candidate who is more skilled than they are; someone wants to recommend their college friend for the role; someone just prefers to work with extraverts but the candidate comes across as an introvert on the phone — whatever it is. Then arbitrary and secondary aspects of a test solution get amplified for debate to steer the decision in a political direction. Nobody wants to believe their teammates would do this, and many times people do it without realizing. But I’ve seen my own team doing it in the past during interview evaluations, and it’s particularly bad when debating over how to assess a take-home test submission. As a candidate I experienced it a few times too, notably with Asana as mentioned before. The other sad part of that Asana interview was that nobody in any part of the Asana pipeline ever asked me about my work experience. Every interview was tech questions, programming questions, and the take-home test. Not a single question about my work history. In the test feedback, the reviewer suggested I might not work well in a product-oriented environment — yet nobody had asked about my work experience in a product-oriented environment at an education technology company. That’s just one example, but there were others. It saddens me a lot that the tech industry is like this. It’s so myopic to believe I could assess someone’s subjective disposition towards being results-oriented or customer-focused or product-minded based on some random 3-hour test, and never even so much as ask them about their actual work history, where they actually solved problems and delivered products. It doesn’t feel good to be told, “based on this ambiguous code test, you’re probably not good at thinking about product development,” when you’ve got plenty of product-development items on your resume you’d be happy to share yet are never even asked about.

erinakinci 8y ago
I continue to agree that it doesn't sound like this was a good experience. I think you'd find that our data science interview process is actually substantially different today and changes many of the things that felt frustrating to you when you went through it a few years ago, including an increased focus on discussing past experience. Thank you again for your candid feedback.