4 ms·

The counter argument is how does this provide a level playing ground upon which to compare candidates? Some days, what a team works on is grunt work or trivial.

The counter argument is how does this provide a level playing ground upon which to compare candidates? Some days, what a team works on is grunt work or trivial. Some days it is designing new, high scale software. And some days it it working on something requiring esoteric knowledge or cracking a difficult bug. There is so much variability depending on when the candidate comes in. Compare this to a set of interview questions or tasks that touch on how an applicant deals with many facets of the above, which is what I'm familiar with.

mgkimsal 9y ago
the job isn't just about the candidate - it's also about the existing team, and the pairing together provides data points for both parties as to how they'd work together, regardless of the task at hand.
learc83 9y ago
>Compare this to a set of interview questions or tasks that touch on how an applicant deals with many facets of the above, which is what I'm familiar with. And none of those interview questions will measure how the candidate performs in your team's day to day environment, nearly as well.
im_down_w_otp 9y ago
I would question what signaling effects you think you're actually measuring when you're presenting contrived scenarios for which you already know the answers. There's a tremendous amount of variability in whether or not the artificial test that's been created actually maps appropriately to what you're trying to assess. It's extremely difficult to do well. There's further variability in the proctoring and posturing in how the assessment is performed, not just the assessment itself. Almost certainly you've added layers of obfuscation to the thing you actually want to know, unless what you want to know is how good the person is at taking your specific test, which may or may not usefully test anything, and which may or may not be administered in a way that could get useful results even if the test itself is meticulously well-crafted. If you're intent on, or can't avoid, operating a more traditional looking hiring process, then I'd suggest at least trying this. Even if the assessment is contrived, then use questions that nobody in the room knows the answers to, including yourself. It doesn't relate to the work directly necessarily, but it can at least help mitigate the effects of survivorship bias.

mgkimsal 9y ago

the job isn't just about the candidate - it's also about the existing team, and the pairing together provides data points for both parties as to how they'd work together, regardless of the task at hand.

learc83 9y ago

>Compare this to a set of interview questions or tasks that touch on how an applicant deals with many facets of the above, which is what I'm familiar with. And none of those interview questions will measure how the candidate performs in your team's day to day environment, nearly as well.

im_down_w_otp 9y ago

I would question what signaling effects you think you're actually measuring when you're presenting contrived scenarios for which you already know the answers. There's a tremendous amount of variability in whether or not the artificial test that's been created actually maps appropriately to what you're trying to assess. It's extremely difficult to do well. There's further variability in the proctoring and posturing in how the assessment is performed, not just the assessment itself. Almost certainly you've added layers of obfuscation to the thing you actually want to know, unless what you want to know is how good the person is at taking your specific test, which may or may not usefully test anything, and which may or may not be administered in a way that could get useful results even if the test itself is meticulously well-crafted. If you're intent on, or can't avoid, operating a more traditional looking hiring process, then I'd suggest at least trying this. Even if the assessment is contrived, then use questions that nobody in the room knows the answers to, including yourself. It doesn't relate to the work directly necessarily, but it can at least help mitigate the effects of survivorship bias.