4 ms·

ska, our aim with the challenge was to allow candidates to not be biased by a fixed outcome and try to solve the problem as they would solve any real data scien

ska, our aim with the challenge was to allow candidates to not be biased by a fixed outcome and try to solve the problem as they would solve any real data science problem. This meant we couldn't automatically score/rank a candidate's solution. We do provide them with an evaluation metric in the problem description (Mean Absolute Error). Here is scoring rubric we provide to the interviewers when they review the submission - https://d.pr/i/hNYY0u https://d.pr/i/hNYY0u Would love to hear more opinions on our scoring rubric

ska 7y ago
That's useful information, thanks. I'll add some thoughts on the rubric. [edit]. Initial thoughts: - "data wrangling" scoring difficult given this task - more weight to "rationale", that's more important the "performance", here. - not enough focus on communication capabilities - really need something on validation - "proficiency" measure you use is pretty much impossible to accurately evaluate from your example question - way too much weight to modeling section overall
- shikharja 7y ago
  By 3rd bullet, you mean Data Validation?
  - ska 7y ago
    both data validation and model validation are important. Of course more abstractly you want to validate the entire process - but I was referring to those two, as it is hard to see how to address the latter in a format like this.