3 ms·
I would agree with this in that the "audition project" would not be done by top candidates - but I'd like to suggest that they already do the audition project.
by codemac 13y ago
I would agree with this in that the "audition project" would not be done by top candidates - but I'd like to suggest that they already do the audition project.
The group I'm in (all flash SAN storage) has gone from single digits to ~80 people in under a year, and basically the only predictive metric is referrals from people who have made other good referrals. We're writing a complete HPC OS, and it's sink or swim. The referrals are almost all swimming.
That's it. Almost all other hires have been completely random in quality.
So these amazing engineers that you're really struggling to hire are actually being hired because of a career long audition with their peers. It's wildly incestuous, and has some interesting effects. Some effects are bad: no one knows new tools/new development styles. Some effects are good: at the top, gender split is much closer to 50/50.
I'd love to figure out how to vet someone in under 3 months that is not part of this professional tribe, but even typing that out I'm almost suspicious they'd succeed and become part of it if they aren't in the tribe already. It seems like even my attitudes are a problem in opening up these kinds of candidate searches.
- tptacek 13y agoYou make an interesting macro point. But there's a world of difference between an entry level full-time job and a 1-2 week "audition project". The phenomenon you're describing is what makes recruiting such an interesting problem. The largest or wealthiest employers can often afford to constrain their candidates to people who have proven themselves in other settings. Which of course leaves a "Moneyball"-type opening to the employers who can figure out how to hire for aptitude instead of experience. Which makes hiring at those savvier employees a high-stakes, high-value process, instead of a tedious formality inflicted across the whole team.
- codemac 13y agoThe point I was trying to make wasn't about hiring "for experience" but leveraging those who were around them to vouch for their aptitude. This is a distinct difference, and you have to work hard to get people to think in these terms instead of amount of experience. I may be naive in even thinking we're accomplishing this. Without the public displays of performance that baseball has, there's another level of indirection that you must filter through to try and illustrate a candidates performance. This makes it something like the "Moneyball of Moneyball of programming".
- thaumasiotes 13y ago> Which of course leaves a "Moneyball"-type opening to the employers who can figure out how to hire for aptitude instead of experience. I want to call for tokenadult's canned "within the US, look for a work sample. Outside the US, use an IQ test" post. We figured out _how_ to hire for aptitude a long time ago (subject, of course, to the fact that prediction is never perfect); Malcom McLean apparently filled his new shipping company top-down by who, among the people who worked for his old trucking company, scored highest on an IQ test. But then we decided that that would be illegal in the future?
- tptacek 13y agoWe know what the solution looks like, but not what its precise measurements are; the person who comes up with a universal, economical, predictively powerful work sample test for software developers is probably going to make a lot of money.
- tel 13y agoReally a family of such metrics because the moment the first one hits the ground in any earnestly it'll be gamed to death. Nothing ruins a powerful, economical, repeatable indicator for predicting economic activity faster than using it.
- ecspike 13y agoEven worse is the unpaid audition project. I've had a prospective employer try to pitch me on that and they were shocked when I balked.