3 ms·
>For each resume, I had a pretty good idea of how strong the engineer in question was, and I split resumes into two strength-based groups. To make this judgment
by hacknat 12y ago
>For each resume, I had a pretty good idea of how strong the engineer in question was, and I split resumes into two strength-based groups. To make this judgment call, I drew on my personal experience — most of the resumes came from candidates I placed (or tried to place) at top-tier startups. In these cases, I knew exactly how the engineer had done in technical interviews, and, more often than not, I had visibility into how they performed on the job afterwards. The remainder of resumes came from engineers I had worked with directly.
Does anyone else think this is a huge caveat getting thrown into a blender of numbers? Why does this person's opinion even matter? Why should I take this at face value? I'm not trying to say resumes are a good indicator of anything, but neither is this "study".
- usaar333 12y agoAgreed. The study is showing how much people's views of resume X agrees with the author's full evaluation. But what I or my company consider a quality candidate may differ from the author. On another note, I've found that for recent grads, GPA, school, and major (all items on the resume) are excellent predictors of interview performance.
- itsdrewmiller 12y agoIt goes further than just that - the stuff about Fleiss' kappa shows that not only do these people not agree with the author; they also don't agree with each other. Maybe all development shops are so different that there is no significant correlation between success at one place and success at another, but I highly doubt it!
- apolretom 12y agoThank you. This entire 'study' is nonsense because this recruiter has no idea how 'strong' these candidates really are. She only thinks she knows, just like every arrogant jerk who's ever conducted a technical interview.
- cowls 12y agoYep, nonsense in = nonsense out. This "study" is pointless.
- AmirS2 12y agoLooks like the author explicitly considered that: > To try to understand whether people really were this bad at the task or whether perhaps the task itself was flawed, I ran some more stats. One thing I wanted to understand, in particular, was whether inter-rater agreement was high. In other words, when rating resumes, were participants disagreeing with each other more often than you’d expect to happen by chance? If so, then even if my criteria for whether each resume belonged to a strong candidate wasn’t perfect, the results would still be compelling The result of the Fleiss' kappa test subsequently run was negative, i.e. people didn't agree with each other either. So maybe the author's judgement was wrong, but that doesn't affect the conclusions.