4 ms·

Are you saying there's no such thing as developers significantly better than the norm, or rather that it's infeasible to identify them? I don't think your poin

by dsacco 8y ago

Are you saying there's no such thing as developers significantly better than the norm, or rather that it's infeasible to identify them? I don't think your point about precisely quantifying the definition of a "great developer" is a good one. I can capture an observation of superior capability in several dimensions without having to rigorously quantify the relative differences in capability or precisely why one is more capable than the other. If one developer accomplishes in a few days what takes another two weeks with code that is at least as maintainable and performant, that developer is better. If that developer is better when put next to most of the other developers you have around you, then they're a "great developer." Your focus on quantifiable definition seems to imply that significant differences in human capital can only be ascertained mathematically, but that doesn't reflect quite a lot that we're already familiar with in everyday life. If I'm placed in front of two walls which are both much taller than I am, I can see which of the two is taller than the other if the difference is significant as long as the tops aren't out of sight. I don't need to quantify this; I could just say, "One looks to be a lot taller than the other, but I'm not sure by how much exactly, or what their respective heights are." I'll happily agree that a lot of tech interviewing (and interviewing in general) is dreadfully broken. I'll also happily agree that figuring out which skills to prioritize and how to judge the level of those skills in candidates is very difficult work. But I strongly disagree that the fundamental premise of that work is intrinsically "voodoo." If your bar is a mathematical formalization then of course you're going to be disappointed, but that also holds true for the majority of our professional and personal activity.

andrewstuart 8y ago
In a sentence, I am saying that pretty much every recruiting process at every company, when it assesses developers, results in a meaningless decision, despite the universal certainty at every company that their recruiting process absolutely definitely results in making accurate decisions about people. Did the interview process find a great developer? Maybe. Did the recruiting process reject great developers? How would you ever know? I'm pretty much certain that most companies reject many more developers who would be really productive/effective, than they choose to hire.
- dsacco 8y ago
  That still seems like a pretty strong claim. Inefficient? Sure. Misaligned incentives? Definitely. But actually meaningless? If a company turns away many developers who would be excellent hires but consistently meets its product/engineering goals and hires more good developers than bad developers overall, their recruiting process certainly has a positive signal. It's obviously inefficient, but it couldn't be meaningless. I also don't think it's fair to characterize most companies as believing their recruiting practices infallible. Google's former director of recruiting has frequently talked about how their recruiting processes (and particularly interviewer feedback) seemed very noisy and fairly inconsistent in a variety of dimensions. The common recruiting-page-positive-vibes spiel might indicate a lot of confidence, but that's not what I would use to determine it. A lot of companies also seem pretty happy to try out new programs like e.g. Triplebyte or otherwise innovate on their sourcing processes.
  - andrewstuart 8y ago
    Not a direct response to your comment, but I once sent someone who I thought was a really bright developer to a company, who assessed him and rejected him. That same developer soon after got a job at Amazon. When I told the original employer who had rejected him, he said "So Amazon are hiring shit people now?". That hiring manager I have never ever known to question any hiring decision he ever made - he has always been certain that the rejection was correct. And what I have observed in recruiting is that any rejection decision made by a company is made with 110% certainty.
  - jaggederest 8y ago
    > If a company turns away many developers who would be excellent hires but consistently meets its product/engineering goals and hires more good developers than bad developers overall, their recruiting process certainly has a positive signal. It's obviously inefficient, but it couldn't be meaningless. Actually, that result would be precisely what you would expect from a process that fails to disprove the null hypothesis, and is therefore meaningless. You'd see them hiring essentially a random sample of engineers with the same distribution as the whole population. I leave it to someone better at statistics than I am to figure out what an actually effective hiring process looks like, but I think the correlation between "is a relatable person" and hiring is much higher than "is a good engineer".
gdltec 8y ago
A developer becomes a "great developer" when the company, team, resources, projects, recognition, etc., are compatible with that person. Under that logic, I believe any programmer can be great if they desire to do so and find the environment and motivation to thrive. Most technical interviews fail to find the right people because interviewers and hiring managers usually go at it with an "idea" of what a "great developer" looks like to them. In most cases, everyone ends up hiring people who don't work out and miss out on people who could have become the "great developers" there were looking for in the first place.
- brandall10 8y ago
  I agree. I have a strong belief that the best way to interview a candidate is to take a literal real world problem the team experienced, distill it down to its essence so it can be tackled in a 2-4 hour pairing session. You should have full access to internet, whiteboard, etc. If the candidate is having to review CS algos and doing whiteboard prep, you're doing it wrong (unless that stuff truly is what the job entails). If the candidate is nervous because they feel like they're giving a dissertation, you're doing it wrong. It should mimic as closely as possible what the actual job will be like. You're not going to fully know whether or not they will produce in practice, but you'll be able to tell that you can work with them and that's half the battle.
- ubernostrum 8y ago
  I will second this because I've seen it happen, both to me and to other people -- an environment that wasn't a good fit changes and suddenly someone excels, or someone excels and the environment changes and suddenly it's like they can't do anything right. Also, having clear criteria set up in advance is essential. You should know before you go into the room (or pick up the phone, or whatever) not only what constitutes a pass versus a fail, but also what kinds of things in the interview signal more than just passing. And the pass/fail can't just be "regurgitated an algorithm we wanted them to regurgitate", because that tells you nothing about whether somebody's actually good to work with.

by dsacco 8y ago

andrewstuart 8y ago

In a sentence, I am saying that pretty much every recruiting process at every company, when it assesses developers, results in a meaningless decision, despite the universal certainty at every company that their recruiting process absolutely definitely results in making accurate decisions about people. Did the interview process find a great developer? Maybe. Did the recruiting process reject great developers? How would you ever know? I'm pretty much certain that most companies reject many more developers who would be really productive/effective, than they choose to hire.

dsacco 8y ago
That still seems like a pretty strong claim. Inefficient? Sure. Misaligned incentives? Definitely. But actually meaningless? If a company turns away many developers who would be excellent hires but consistently meets its product/engineering goals and hires more good developers than bad developers overall, their recruiting process certainly has a positive signal. It's obviously inefficient, but it couldn't be meaningless. I also don't think it's fair to characterize most companies as believing their recruiting practices infallible. Google's former director of recruiting has frequently talked about how their recruiting processes (and particularly interviewer feedback) seemed very noisy and fairly inconsistent in a variety of dimensions. The common recruiting-page-positive-vibes spiel might indicate a lot of confidence, but that's not what I would use to determine it. A lot of companies also seem pretty happy to try out new programs like e.g. Triplebyte or otherwise innovate on their sourcing processes.
- andrewstuart 8y ago
  Not a direct response to your comment, but I once sent someone who I thought was a really bright developer to a company, who assessed him and rejected him. That same developer soon after got a job at Amazon. When I told the original employer who had rejected him, he said "So Amazon are hiring shit people now?". That hiring manager I have never ever known to question any hiring decision he ever made - he has always been certain that the rejection was correct. And what I have observed in recruiting is that any rejection decision made by a company is made with 110% certainty.
- jaggederest 8y ago
  > If a company turns away many developers who would be excellent hires but consistently meets its product/engineering goals and hires more good developers than bad developers overall, their recruiting process certainly has a positive signal. It's obviously inefficient, but it couldn't be meaningless. Actually, that result would be precisely what you would expect from a process that fails to disprove the null hypothesis, and is therefore meaningless. You'd see them hiring essentially a random sample of engineers with the same distribution as the whole population. I leave it to someone better at statistics than I am to figure out what an actually effective hiring process looks like, but I think the correlation between "is a relatable person" and hiring is much higher than "is a good engineer".

gdltec 8y ago

A developer becomes a "great developer" when the company, team, resources, projects, recognition, etc., are compatible with that person. Under that logic, I believe any programmer can be great if they desire to do so and find the environment and motivation to thrive. Most technical interviews fail to find the right people because interviewers and hiring managers usually go at it with an "idea" of what a "great developer" looks like to them. In most cases, everyone ends up hiring people who don't work out and miss out on people who could have become the "great developers" there were looking for in the first place.

brandall10 8y ago
I agree. I have a strong belief that the best way to interview a candidate is to take a literal real world problem the team experienced, distill it down to its essence so it can be tackled in a 2-4 hour pairing session. You should have full access to internet, whiteboard, etc. If the candidate is having to review CS algos and doing whiteboard prep, you're doing it wrong (unless that stuff truly is what the job entails). If the candidate is nervous because they feel like they're giving a dissertation, you're doing it wrong. It should mimic as closely as possible what the actual job will be like. You're not going to fully know whether or not they will produce in practice, but you'll be able to tell that you can work with them and that's half the battle.
ubernostrum 8y ago
I will second this because I've seen it happen, both to me and to other people -- an environment that wasn't a good fit changes and suddenly someone excels, or someone excels and the environment changes and suddenly it's like they can't do anything right. Also, having clear criteria set up in advance is essential. You should know before you go into the room (or pick up the phone, or whatever) not only what constitutes a pass versus a fail, but also what kinds of things in the interview signal more than just passing. And the pass/fail can't just be "regurgitated an algorithm we wanted them to regurgitate", because that tells you nothing about whether somebody's actually good to work with.