4 ms·

I am sure we agree that it is not random for many other professions. Why should the filter almost random for SWEs? What's so special about them? Let's say you

I am sure we agree that it is not random for many other professions. Why should the filter almost random for SWEs? What's so special about them? Let's say you are asked to rate a boxer. You have their resume (who and when they fought) and you can "interview" (no sparring partner, even) them for 10 hours. If I am going to rate them on a scale 1-10, I could certainly rate them with an error of +-10% (on average), that is I could make an accurate prediction of their "worth" in the next year. It is not just the "interview" without sparring partners, it is the interview + resume. Only the interview and no resume, well, there have been a lot of promising boxers who went nowhere, and in fact it is much more challenging to evaluate juniors than seniors. In my experience, I was able to rate SWE and ML types pretty accurately, conditioned on the fact that they were hired (and they hired some people I rejected and some I approved). I had them for one hour and I read the resume. I think the secret was that I asked good questions.

brundolf 4y ago
Because software engineering isn't boxing? It's hard to separate a team's performance from the rest of the org, and it's hard to separate an individual's performance from the rest of their team. And even if you could- how do you measure success? Unlike boxing, there's no binary win/loss condition. KPIs? But what about an engineer who does a brilliant job quietly making sure a piece of software will be maintainable over the next decade? How do you measure that, even from inside the organization, much less outside of it? And if you could, how do you know that success in one org (with a particular codebase, stack, team culture, velocity, etc) translates to success in a different one? And then because of all the above: even a resume doesn't tell you very much. It tells you they convinced other employers they were worth hiring/keeping, but those other employers were subject to the same imprecisions/measurement problems that you are. It's very possible they were all wrong about the quality of this person's contributions. Software development is a deeply human activity that's impossible to reduce to simplified signifiers. It's my opinion that the only way to evaluate an engineer is to be another engineer working alongside them for an extended time. And even then- that doesn't necessarily translate to other scenarios they might work in. Another commenter here said they do a 30-day trial period with new engineers, to see if they're a good fit. From a hiring perspective, this is the only process I can think of that would be decently accurate. Though obviously it would be pretty rough on the engineer to not know if they have job security for 30 days.

brundolf 4y ago

Because software engineering isn't boxing? It's hard to separate a team's performance from the rest of the org, and it's hard to separate an individual's performance from the rest of their team. And even if you could- how do you measure success? Unlike boxing, there's no binary win/loss condition. KPIs? But what about an engineer who does a brilliant job quietly making sure a piece of software will be maintainable over the next decade? How do you measure that, even from inside the organization, much less outside of it? And if you could, how do you know that success in one org (with a particular codebase, stack, team culture, velocity, etc) translates to success in a different one? And then because of all the above: even a resume doesn't tell you very much. It tells you they convinced other employers they were worth hiring/keeping, but those other employers were subject to the same imprecisions/measurement problems that you are. It's very possible they were all wrong about the quality of this person's contributions. Software development is a deeply human activity that's impossible to reduce to simplified signifiers. It's my opinion that the only way to evaluate an engineer is to be another engineer working alongside them for an extended time. And even then- that doesn't necessarily translate to other scenarios they might work in. Another commenter here said they do a 30-day trial period with new engineers, to see if they're a good fit. From a hiring perspective, this is the only process I can think of that would be decently accurate. Though obviously it would be pretty rough on the engineer to not know if they have job security for 30 days.