No one has figured this out-- not for programmers and not for any other profession. It's certainly possible for individuals to develop good or bad reputations,

No one has figured this out-- not for programmers and not for any other profession. It's certainly possible for individuals to develop good or bad reputations, but to be able to accurately evaluate how anyone will do in a job in the absence of other data is very much hit-or-miss. The most elite orgs might LOOK LIKE they figured it out, but how much of that is really their process vs just being able to attract talent because of compensation or reputation? How much of it is more a matter of being able to churn through people and replace people that get hired but can't hack it once they're in? How much of it is skillful use of professional referrals? > It would be very interesting with a clean "we asked n programmers to solve this"-type problem [...] It's been done, right? There's all those whiteboard interviews that highly resourced orgs study endlessly. There's a whole industry around preparing for these things. What do we have to show for it as an industry? Not much. If the stories are to be believed, folks are still flunking "fizz-buzz" at alarming rates.