6 ms·

Having hired dozens of devs, I can confidently say there is absolutely 0% chance to consistently successfully identify good developers in any reasonable amount

Having hired dozens of devs, I can confidently say there is absolutely 0% chance to consistently successfully identify good developers in any reasonable amount of interviewing / assessment time period. The best way is someone brings in an existing code portfolio and discusses it. The second best way is someone completes multiple design and development exercises of varying complexity, constraints, and use cases. The third best way is they complete a single exercise and provide commentary on alternative designs. There is no fourth best way; all other approaches are essenrially stochastic and select for interviewing traits not development traits. The actual best method I think is a 3 month probationary period which is more or less an extended interview. They're asked to contribute to existing codebases, participate in code review, go through some architecture design sessions, conduct stakeholder interviews - things that again are mostly impossible to accurately gauge in a typical candidate assessment window. By the way a tremendous book for interviewers and hiring managers is How Judges Think by Richard Posner. A lot of it applies to hiring, and he's a great writer.

triceratops 7y ago
> The actual best method I think is a 3 month probationary period which is more or less an extended interview Most states in the US have at-will employment terms. Probationary periods are common in other countries. And yet no one seems to want to do what you're suggesting.
- UnpossibleJim 7y ago
  The problem is, most companies in the US want to hire you on as a Contractor and not a full time employee for those three months. I can't speak for everyone (but most folks, I assume), but I need medical and dental. I can't go three months for the chance of maybe getting it. I can't do it personally and i can't do that sort of thing to my family. there are other options in other countries (depending on the country). I'm a centrist by nature, but this makes a "three month interview" untenable, IMO.
  - triceratops 7y ago
    Sorry what I meant was US companies could treat the first 3 months of any full-time/W-2 employee as probationary because they can fire anyone for any reason other than protected characteristics. It doesn't even need to be an official company policy documented anywhere.`
    - DavidWoof 7y ago
      They could in theory, but nobody wants to. I used to see probationary periods, but I haven't seen a company use that for salaried staff for a long time. HR and Legal have put an end to that kind of thing. So Jim is right, in practice the only way HR is going to let you have a probationary period is to use contractors, and that limits your hiring pool significantly.
      - Consultant32452 7y ago
        Every decent sized contracting firm I've ever worked with offered benefits. It's more expensive, but you just price that into your rate. So I don't understand what the concern is. Is it the risk that after 3 months you won't be brought on full time?
        DavidWoof 7y ago
        Upheaval is the issue. First, most contracting firms have pretty crappy medical plans. Secondly, there's generally a wait period before full benefits kick in. Thirdly, you wind up switching your medical plan multiple times in a short period. That's fine if you're single, but if you've got kids that might mean switching doctors multiple times, which is a gigantic huge deal. Also, you have multiple periods of time where you don't really know your benefits which is terrifying. And if the company does drop you after three months for whatever reason, now you're doing it again. And keep in mind it's definitely a seller's market if you're a good dev. It's incredibly unlikely that the 3-month contract is the only offer on the table. So all things being equal, who would take it?
- deleted 7y ago
  [deleted]
DavidWoof 7y ago
#1 is great, but impossible for a lot of people due to NDAs. The 3-month probationary period is indeed the only way to really evaluate devs, but I'm not so sure it's a good way to hire them. After 3 months, the mediocre devs have friends, etc., which makes it hard to get rid of them without creating a weird morale issue and massive team disruptions. And then you have to add in the limiting effect on the hiring pool (though I wonder if that really matters, I suspect good devs know who they are). Number 3 is my favorite, but there's a trade-off: almost anything large enough to allow for multiple designs is probably too large to require all applicants to complete. But I also love having a candidate discuss their own code. I'm really curious what types of exercises you have for this kind of thing, and where it fits in the interview process?
- AmericanChopper 7y ago
  > After 3 months, the mediocre devs have friends, etc., which makes it hard to get rid of them without creating a weird morale issue and massive team disruptions I’ve seen this mistake happen so many times. Especially in early stage start ups. If you want morale issues and disruptions, you’ll achieve it 100% of the time by retaining underperforming staff. Even worse, this ultimately leads to your best team members leaving if it’s an issue you can’t solve. Fail fast applies to HR too, you need to learn to fire fast.
  - bradleyjg 7y ago
    It's true that keeping on terrible employees is worse than firing them, but that's the answer to a different question. The question at hand is should you opt for a system you know is going to involve a lot of firing fast, or do your best to avoid bad hires even at the cost of a higher false negative rate?
    - AmericanChopper 7y ago
      Hiring and firing are expensive in a whole lot of different ways, but it takes time and effort to get right, and you’ll never get it right every time. So actually using the probationary period properly, and getting good at firing people are things you’ll have to do anyway.
closeparen 7y ago
How often are you hiring people whose professional work is open source? How often are you hiring people for roles where their professional work will be open source?
- habitue 7y ago
  the OP isn't saying it's always possible, just that it's the best way if it is possible
akelly 7y ago
Nobody does probationary periods because few candidates would pick a probationary period offer over a standard job offer. And it's presumed that top level talent will have multiple offers on the table, and if you don't then there's something wrong with you, so having a probationary period selects for lower quality candidates.
- EliRivers 7y ago
  I would assume that a company without probationary periods is lower quality; either the work is so uniform that anyone can do it and employees are fungible, or they simply accept bad employees and live with them. Every company I've worked for has had some kind of probationary period. I don't live or work in the US, though; we can dismiss someone during their probationary period with ease, but after that they have workplace protections. If we could fire anyone at anytime for anything we like, that effectively makes the entire employment period a probationary period. Probation never ends if you can be fired at any time.
EliRivers 7y ago
The actual best method I think is a 3 month probationary period My workplace has a six month probationary period; I brought in a mandatory three month review after watching one group screw this up badly. At the end of the six months, the employee came in to work all happy as usual thinking everything was great, and was promptly fired. The three month review is the point at which the employee is told that they're on track, or they're below standard. If on track, just keep going the same way and if they don't get taken aside for a specific chat in the following three months, told to assume that they're going to pass the probation period; we have definitely turned probation failures into probation successes via this mid-point review. It's also their opportunity to tell the company what the company is doing wrong; what the company is doing that will make them choose not to stay. This too has happened, and we have retained good employees by listening to them at the three month point and making changes. If they're below standard, they're told what they need to improve and are offered help to improve, or they can just sack it now and walk (or, as happened once and once only so far, they're considered unrecoverable and we take a long hard look at how that person was hired). The principle we subscribe to is that if the employee is surprised by the results of their probation period, that employee's team lead and by association "the company" has really screwed up. If an employee doesn't know how they're doing after six months on the job, something has gone very wrong.
boltzmann_brain 7y ago
Actually there is a better way assuming a candidate with experience: discuss what they did, ask questions like: "why did you chose this solution?", "what were the main challenges you encountered?", "how would you change your solution in hindsight?", etc. There is no way to bullshit through this or such attempt will be quickly clearly apparent.

triceratops 7y ago

> The actual best method I think is a 3 month probationary period which is more or less an extended interview Most states in the US have at-will employment terms. Probationary periods are common in other countries. And yet no one seems to want to do what you're suggesting.

UnpossibleJim 7y ago
The problem is, most companies in the US want to hire you on as a Contractor and not a full time employee for those three months. I can't speak for everyone (but most folks, I assume), but I need medical and dental. I can't go three months for the chance of maybe getting it. I can't do it personally and i can't do that sort of thing to my family. there are other options in other countries (depending on the country). I'm a centrist by nature, but this makes a "three month interview" untenable, IMO.
- triceratops 7y ago
  Sorry what I meant was US companies could treat the first 3 months of any full-time/W-2 employee as probationary because they can fire anyone for any reason other than protected characteristics. It doesn't even need to be an official company policy documented anywhere.`
  - DavidWoof 7y ago
    They could in theory, but nobody wants to. I used to see probationary periods, but I haven't seen a company use that for salaried staff for a long time. HR and Legal have put an end to that kind of thing. So Jim is right, in practice the only way HR is going to let you have a probationary period is to use contractors, and that limits your hiring pool significantly.
    - Consultant32452 7y ago
      Every decent sized contracting firm I've ever worked with offered benefits. It's more expensive, but you just price that into your rate. So I don't understand what the concern is. Is it the risk that after 3 months you won't be brought on full time?
      - DavidWoof 7y ago
        Upheaval is the issue. First, most contracting firms have pretty crappy medical plans. Secondly, there's generally a wait period before full benefits kick in. Thirdly, you wind up switching your medical plan multiple times in a short period. That's fine if you're single, but if you've got kids that might mean switching doctors multiple times, which is a gigantic huge deal. Also, you have multiple periods of time where you don't really know your benefits which is terrifying. And if the company does drop you after three months for whatever reason, now you're doing it again. And keep in mind it's definitely a seller's market if you're a good dev. It's incredibly unlikely that the 3-month contract is the only offer on the table. So all things being equal, who would take it?
deleted 7y ago
[deleted]

DavidWoof 7y ago

#1 is great, but impossible for a lot of people due to NDAs. The 3-month probationary period is indeed the only way to really evaluate devs, but I'm not so sure it's a good way to hire them. After 3 months, the mediocre devs have friends, etc., which makes it hard to get rid of them without creating a weird morale issue and massive team disruptions. And then you have to add in the limiting effect on the hiring pool (though I wonder if that really matters, I suspect good devs know who they are). Number 3 is my favorite, but there's a trade-off: almost anything large enough to allow for multiple designs is probably too large to require all applicants to complete. But I also love having a candidate discuss their own code. I'm really curious what types of exercises you have for this kind of thing, and where it fits in the interview process?

AmericanChopper 7y ago
> After 3 months, the mediocre devs have friends, etc., which makes it hard to get rid of them without creating a weird morale issue and massive team disruptions I’ve seen this mistake happen so many times. Especially in early stage start ups. If you want morale issues and disruptions, you’ll achieve it 100% of the time by retaining underperforming staff. Even worse, this ultimately leads to your best team members leaving if it’s an issue you can’t solve. Fail fast applies to HR too, you need to learn to fire fast.
- bradleyjg 7y ago
  It's true that keeping on terrible employees is worse than firing them, but that's the answer to a different question. The question at hand is should you opt for a system you know is going to involve a lot of firing fast, or do your best to avoid bad hires even at the cost of a higher false negative rate?
  - AmericanChopper 7y ago
    Hiring and firing are expensive in a whole lot of different ways, but it takes time and effort to get right, and you’ll never get it right every time. So actually using the probationary period properly, and getting good at firing people are things you’ll have to do anyway.

closeparen 7y ago

How often are you hiring people whose professional work is open source? How often are you hiring people for roles where their professional work will be open source?

habitue 7y ago
the OP isn't saying it's always possible, just that it's the best way if it is possible

akelly 7y ago

Nobody does probationary periods because few candidates would pick a probationary period offer over a standard job offer. And it's presumed that top level talent will have multiple offers on the table, and if you don't then there's something wrong with you, so having a probationary period selects for lower quality candidates.

EliRivers 7y ago
I would assume that a company without probationary periods is lower quality; either the work is so uniform that anyone can do it and employees are fungible, or they simply accept bad employees and live with them. Every company I've worked for has had some kind of probationary period. I don't live or work in the US, though; we can dismiss someone during their probationary period with ease, but after that they have workplace protections. If we could fire anyone at anytime for anything we like, that effectively makes the entire employment period a probationary period. Probation never ends if you can be fired at any time.

EliRivers 7y ago

The actual best method I think is a 3 month probationary period My workplace has a six month probationary period; I brought in a mandatory three month review after watching one group screw this up badly. At the end of the six months, the employee came in to work all happy as usual thinking everything was great, and was promptly fired. The three month review is the point at which the employee is told that they're on track, or they're below standard. If on track, just keep going the same way and if they don't get taken aside for a specific chat in the following three months, told to assume that they're going to pass the probation period; we have definitely turned probation failures into probation successes via this mid-point review. It's also their opportunity to tell the company what the company is doing wrong; what the company is doing that will make them choose not to stay. This too has happened, and we have retained good employees by listening to them at the three month point and making changes. If they're below standard, they're told what they need to improve and are offered help to improve, or they can just sack it now and walk (or, as happened once and once only so far, they're considered unrecoverable and we take a long hard look at how that person was hired). The principle we subscribe to is that if the employee is surprised by the results of their probation period, that employee's team lead and by association "the company" has really screwed up. If an employee doesn't know how they're doing after six months on the job, something has gone very wrong.

boltzmann_brain 7y ago

Actually there is a better way assuming a candidate with experience: discuss what they did, ask questions like: "why did you chose this solution?", "what were the main challenges you encountered?", "how would you change your solution in hindsight?", etc. There is no way to bullshit through this or such attempt will be quickly clearly apparent.