4 ms·

The trouble is, this "je ne sais quoi" that you speak of, that you "could tell he had the chops to figure things out", is unquantifiable and almost totally sub

The trouble is, this "je ne sais quoi" that you speak of, that you "could tell he had the chops to figure things out", is unquantifiable and almost totally subjective. To be a bit blunt and cynical, this way of evaluating candidates basically boils down to, "do I like this candidate?" Your criteria for liking someone happens to be that you look for these nebulous "chops for figuring things out". What about the other interviewers at your company? What kind of stuff would they like? Would they evaluate "chops" the same way you do? Or, just imagine that you're interviewing a hypothetically equivalent candidate who also had the same drive and "chops to figure things out", but who happened to be less fluent in conversational English. Are you so sure that you'd be able to pick up on the same underlying "signal" about their chops, if it was obscured by some language fluency noise? Hiring is a Hard Problem, and there's a big gap between well-intentioned intuition and translating that into robust processes.

wbsss4412 5y ago
There is something to be said about the value of human intuition, though. While it may be fallible, part of what makes hiring a hard problem is that many of the things that distinguish good candidates just aren’t objectively measurable.
- pessimizer 5y ago
  > There is something to be said about the value of human intuition, IMO, the something to be said about it is that people have a good intuition about people who remind them of their family, friends, and celebrities they admire; and a bad intuition about people who don't, and remind them of people seen as criminals or bad guys. Obliquely related: http://www.skinema.com/evil/ http://www.skinema.com/evil/
Rd6n6 5y ago
> unquantifiable and almost totally subjective That’s what intuition and judgment calls are for. What does your intuition say is a good way to approach hiring? Does it maybe tell you to figure out relevant skills and ask the candidate about those? Yet if you ignore that voice, you end up timing how long it takes them to invert binary trees from memory instead Everybody on earth who is good at what they do relies utterly on their carefully trained intuition
stuff4ben 5y ago
oh no doubt, getting around implicit bias and all that is hard and is training I've been through and try not to let get in the way. I've interviewed hundreds of people over the course of my career and I've had the greatest success in evaluating the soft skills of an engineer rather than their stated resume skills. For experienced engineers, I still ask technical questions to make sure they're not BSing their way in and have some knowledge of development. But college grads I really want someone with those soft skills. Yes, it is slightly subjective, but not unquantifiable. You have to have shown me times you've done those things, not just say them.
Kranar 5y ago
Exactly, it's so sad hearing someone not realize that their hiring process is basically discriminatory in nature. Can you tell someone had the chops to figure things out if they came from a different cultural background than one you're familiar with? If they had a different social upbringing from yours? If they presented themselves in a way that you personally misunderstood but someone else would have been completely comfortable with? Of course not, OP basically hired someone he got along with, and that's fine on a small scale, but it's not something to pride oneself over. A better approach in my opinion is if you find someone who you get along with, you think they have that kind of fire and passion you're looking for, but they don't know the basics and fundamentals of computer science nor are they familiar with data structures and algorithms, you advise them that if they want to make the most of their potential then they should take some minimal amount of time to really devote themselves to understanding those aspects and then they can reapply for the position at a later date. You can keep in touch with them, even potentially mentor them if you really think they are worth it, but you don't just give someone a pass. Letting someone get by without that knowledge because you like them doesn't really do anyone any favors.
- giantg2 5y ago
  "Exactly, it's so sad hearing someone not realize that their hiring process is basically discriminatory in nature." Yep. I've seen this so much in my company for performance review. It's completely subjective and managers dont even realize when they are discriminating or violating company policies (or laws!).
- 6gvONxR4sf7o 5y ago
  > Exactly, it's so sad hearing someone not realize that their hiring process is basically discriminatory in nature. I've come to the sad conclusion that this is a fundamental tension. Ignoring the pre-interview filtering, our industry has, on one end of the spectrum, a relatively nondiscriminatory process that assumes basically nothing and tests you on as much of the field as they can before hiring. It's nondiscriminatory in that it doesn't matter where you come from as long as you can solve the same structured problems as everyone else. This is the whiteboard interview everyone hates, and it's not all that inclusive. On the other end of the spectrum, we have the fuzzy 'does this person seem like they could become capable quickly and work well with me' process. It's inclusive in that it doesn't require you to be able to solve those stupid problems yet, just be bright enough and hard working enough to learn. But it's not at all unbiased. So do you go for unbiased and give people the same standardized questions, or go for inclusive and have a real conversation? I've found the best people come from the latter process, but I would say that, right? I'm biased.
crispyambulance 5y ago
> Hiring is a Hard Problem, and there's a big gap between well-intentioned intuition and translating that into robust processes. Ultimately, hiring is INTRINSICALLY a subjective process. If it were quantifiable and one could reliably screen based on "objective" criteria, it would have been a solved problem by now and employers could hire "robustly". But it is not and it never will be that way. That said, hiring and assessing talent takes practice and there are some techniques that work better than others. In particular, behavioral techniques are extremely valuable and sadly often take a back seat to persnickety homework quiz questions. It takes some skill and practice to get good at behavioral interviews (for both parties). But when it's used properly, the candidate has a chance to actually demonstrate "chops" even though they may lack some of the job requirements.
- Kranar 5y ago
  Your argument presupposes that an objective criteria is optimal. It's trivial to refute that, for example a company could hire people based on whether they are greater than 6 ft tall, that's an objective criteria but it would still be terrible. Objective doesn't mean "good/optimal/solved" and subjective doesn't mean "bad/suboptimal/unsolved". All subjective/objective refer to is how sensitive a property is to a specific context. An objective measure is not particularly context sensitive whereas a subjective measure is. My company has both objective and subjective criteria. Competency is mostly measured objectively and represents the bulk of our hiring criteria, the subjective assessment involves things like communication, collaboration and professionalism. Both aspects are difficult and imperfect to assess regardless of whether they are objective or subjective, but I don't think it's particularly appropriate to assess someone's competency based on a subjective measure. If someone is a good basketball player, or a good plumber etc... they are likely good regardless of the intricacies of the individual making the assessment that day and hence it's something that is probably best measured objectively. However, whether someone will get along with the rest of the basketball team, or whether a plumber can properly communicate to a customer what the issue is and how to fix it, that may very well depend on what team that player is playing for, or the relationship between that specific plumber and the specific customer. That would be more of a subjective criteria. There certainly are subjective aspects to competency as well as objective aspects to communication, but it's minor (perhaps a basketball player is good only on a specific team, perhaps a plumber is illiterate).
wvenable 5y ago
This mysterious robust process that can deterministically fit humans into the "hire" and "no hire" box doesn't exist. Everything you can describe that's negative about human intuition, I can use to describe whatever robust process you can think of. I want candidates to succeed -- I always start out rooting for them. The interview is not a place to beat people down but it give them opportunities to show their strengths.
Consultant32452 5y ago
I've been responsible for hiring for most of the last 20 years. I've literally never had a false positive where I recommended someone and they performed poorly. I have had one case where I said no, was overruled, and then the person turned out to be awful. To be fair, I have no idea how many false negatives I've had where I said no but they would've been great. I'm always puzzled by people/companies who find this difficult. I don't believe for a second that my intuition is magical. I have a very simple interview process and it works really well. I just ask you what you did, and then I keep asking you increasingly more complex or detailed questions about what you said you did until one of us hits our boundary of knowledge. Then I ask a couple of social questions, usually in the form of "Have you ever dealt with x kind of situation?" Funny anecdote, I asked one guy if he'd ever had a coworker he had personal issues with. He told me some guy made a racist comment and his response was to ask the guy to step outside to fight. We did not hire that guy.
- ryandrake 5y ago
  I always thought a huge blind spot in hiring is that nobody counts or worries about false negatives: Those people who you said "no" to but would have turned out to be great. False positives are often, but not always, easy to spot, and you can know and feel regret about how many hires turned out to be low performers. But it's hard to know or feel regret about the people you turn away. I mean, how would you even track this or correct it? Keep track of everyone you interview, then look at their career 10yrs in the future? Flag people as "no, but maybe we're wrong" in the HR system and then wait for them to re-apply? As an interviewer, I sometimes wonder if I made the right "No Hire" call on that person who might just have been nervous. As a candidate, I always know that when they say "NO" they're wrong :)
  - Consultant32452 5y ago
    I never worry about this. I usually have some idea of what I am looking for: junior, senior, architect. I just want to find someone suitable in an efficient manner. There's tons of solid candidates out there, it's not my responsibility to make sure the ones who get too nervous to communicate during an interview are taken care of.
koonsolo 5y ago
I hired a lot of very good developers, so I was able to rely on my intuition pretty well. Then one time I hired an excellent candidate, technically very strong. Then it turned out he couldn't figure things out on his own. I had a co-interviewer who also thought he was very strong. In the end we had to fire him after a month since he couldn't pick up the pace (his colleague who started at the same time is actually very good) I was "very good at hiring", until I hired the wrong person. Even now it's hard for me to tell what actually went wrong there. It was mostly theory and understanding underlying concepts, but now I definitely mix in a lot of practical issues and examples. Sometimes I think we interviewed his twin brother who could actually program ;). Hiring developers is indeed very difficult. You can only tell when working with them for a few weeks.

wbsss4412 5y ago

There is something to be said about the value of human intuition, though. While it may be fallible, part of what makes hiring a hard problem is that many of the things that distinguish good candidates just aren’t objectively measurable.

pessimizer 5y ago
> There is something to be said about the value of human intuition, IMO, the something to be said about it is that people have a good intuition about people who remind them of their family, friends, and celebrities they admire; and a bad intuition about people who don't, and remind them of people seen as criminals or bad guys. Obliquely related: http://www.skinema.com/evil/ http://www.skinema.com/evil/

Rd6n6 5y ago

> unquantifiable and almost totally subjective That’s what intuition and judgment calls are for. What does your intuition say is a good way to approach hiring? Does it maybe tell you to figure out relevant skills and ask the candidate about those? Yet if you ignore that voice, you end up timing how long it takes them to invert binary trees from memory instead Everybody on earth who is good at what they do relies utterly on their carefully trained intuition

stuff4ben 5y ago

oh no doubt, getting around implicit bias and all that is hard and is training I've been through and try not to let get in the way. I've interviewed hundreds of people over the course of my career and I've had the greatest success in evaluating the soft skills of an engineer rather than their stated resume skills. For experienced engineers, I still ask technical questions to make sure they're not BSing their way in and have some knowledge of development. But college grads I really want someone with those soft skills. Yes, it is slightly subjective, but not unquantifiable. You have to have shown me times you've done those things, not just say them.

Kranar 5y ago

Exactly, it's so sad hearing someone not realize that their hiring process is basically discriminatory in nature. Can you tell someone had the chops to figure things out if they came from a different cultural background than one you're familiar with? If they had a different social upbringing from yours? If they presented themselves in a way that you personally misunderstood but someone else would have been completely comfortable with? Of course not, OP basically hired someone he got along with, and that's fine on a small scale, but it's not something to pride oneself over. A better approach in my opinion is if you find someone who you get along with, you think they have that kind of fire and passion you're looking for, but they don't know the basics and fundamentals of computer science nor are they familiar with data structures and algorithms, you advise them that if they want to make the most of their potential then they should take some minimal amount of time to really devote themselves to understanding those aspects and then they can reapply for the position at a later date. You can keep in touch with them, even potentially mentor them if you really think they are worth it, but you don't just give someone a pass. Letting someone get by without that knowledge because you like them doesn't really do anyone any favors.

giantg2 5y ago
"Exactly, it's so sad hearing someone not realize that their hiring process is basically discriminatory in nature." Yep. I've seen this so much in my company for performance review. It's completely subjective and managers dont even realize when they are discriminating or violating company policies (or laws!).
6gvONxR4sf7o 5y ago
> Exactly, it's so sad hearing someone not realize that their hiring process is basically discriminatory in nature. I've come to the sad conclusion that this is a fundamental tension. Ignoring the pre-interview filtering, our industry has, on one end of the spectrum, a relatively nondiscriminatory process that assumes basically nothing and tests you on as much of the field as they can before hiring. It's nondiscriminatory in that it doesn't matter where you come from as long as you can solve the same structured problems as everyone else. This is the whiteboard interview everyone hates, and it's not all that inclusive. On the other end of the spectrum, we have the fuzzy 'does this person seem like they could become capable quickly and work well with me' process. It's inclusive in that it doesn't require you to be able to solve those stupid problems yet, just be bright enough and hard working enough to learn. But it's not at all unbiased. So do you go for unbiased and give people the same standardized questions, or go for inclusive and have a real conversation? I've found the best people come from the latter process, but I would say that, right? I'm biased.

crispyambulance 5y ago

> Hiring is a Hard Problem, and there's a big gap between well-intentioned intuition and translating that into robust processes. Ultimately, hiring is INTRINSICALLY a subjective process. If it were quantifiable and one could reliably screen based on "objective" criteria, it would have been a solved problem by now and employers could hire "robustly". But it is not and it never will be that way. That said, hiring and assessing talent takes practice and there are some techniques that work better than others. In particular, behavioral techniques are extremely valuable and sadly often take a back seat to persnickety homework quiz questions. It takes some skill and practice to get good at behavioral interviews (for both parties). But when it's used properly, the candidate has a chance to actually demonstrate "chops" even though they may lack some of the job requirements.

Kranar 5y ago
Your argument presupposes that an objective criteria is optimal. It's trivial to refute that, for example a company could hire people based on whether they are greater than 6 ft tall, that's an objective criteria but it would still be terrible. Objective doesn't mean "good/optimal/solved" and subjective doesn't mean "bad/suboptimal/unsolved". All subjective/objective refer to is how sensitive a property is to a specific context. An objective measure is not particularly context sensitive whereas a subjective measure is. My company has both objective and subjective criteria. Competency is mostly measured objectively and represents the bulk of our hiring criteria, the subjective assessment involves things like communication, collaboration and professionalism. Both aspects are difficult and imperfect to assess regardless of whether they are objective or subjective, but I don't think it's particularly appropriate to assess someone's competency based on a subjective measure. If someone is a good basketball player, or a good plumber etc... they are likely good regardless of the intricacies of the individual making the assessment that day and hence it's something that is probably best measured objectively. However, whether someone will get along with the rest of the basketball team, or whether a plumber can properly communicate to a customer what the issue is and how to fix it, that may very well depend on what team that player is playing for, or the relationship between that specific plumber and the specific customer. That would be more of a subjective criteria. There certainly are subjective aspects to competency as well as objective aspects to communication, but it's minor (perhaps a basketball player is good only on a specific team, perhaps a plumber is illiterate).

wvenable 5y ago

This mysterious robust process that can deterministically fit humans into the "hire" and "no hire" box doesn't exist. Everything you can describe that's negative about human intuition, I can use to describe whatever robust process you can think of. I want candidates to succeed -- I always start out rooting for them. The interview is not a place to beat people down but it give them opportunities to show their strengths.

Consultant32452 5y ago

I've been responsible for hiring for most of the last 20 years. I've literally never had a false positive where I recommended someone and they performed poorly. I have had one case where I said no, was overruled, and then the person turned out to be awful. To be fair, I have no idea how many false negatives I've had where I said no but they would've been great. I'm always puzzled by people/companies who find this difficult. I don't believe for a second that my intuition is magical. I have a very simple interview process and it works really well. I just ask you what you did, and then I keep asking you increasingly more complex or detailed questions about what you said you did until one of us hits our boundary of knowledge. Then I ask a couple of social questions, usually in the form of "Have you ever dealt with x kind of situation?" Funny anecdote, I asked one guy if he'd ever had a coworker he had personal issues with. He told me some guy made a racist comment and his response was to ask the guy to step outside to fight. We did not hire that guy.

ryandrake 5y ago
I always thought a huge blind spot in hiring is that nobody counts or worries about false negatives: Those people who you said "no" to but would have turned out to be great. False positives are often, but not always, easy to spot, and you can know and feel regret about how many hires turned out to be low performers. But it's hard to know or feel regret about the people you turn away. I mean, how would you even track this or correct it? Keep track of everyone you interview, then look at their career 10yrs in the future? Flag people as "no, but maybe we're wrong" in the HR system and then wait for them to re-apply? As an interviewer, I sometimes wonder if I made the right "No Hire" call on that person who might just have been nervous. As a candidate, I always know that when they say "NO" they're wrong :)
- Consultant32452 5y ago
  I never worry about this. I usually have some idea of what I am looking for: junior, senior, architect. I just want to find someone suitable in an efficient manner. There's tons of solid candidates out there, it's not my responsibility to make sure the ones who get too nervous to communicate during an interview are taken care of.

koonsolo 5y ago

I hired a lot of very good developers, so I was able to rely on my intuition pretty well. Then one time I hired an excellent candidate, technically very strong. Then it turned out he couldn't figure things out on his own. I had a co-interviewer who also thought he was very strong. In the end we had to fire him after a month since he couldn't pick up the pace (his colleague who started at the same time is actually very good) I was "very good at hiring", until I hired the wrong person. Even now it's hard for me to tell what actually went wrong there. It was mostly theory and understanding underlying concepts, but now I definitely mix in a lot of practical issues and examples. Sometimes I think we interviewed his twin brother who could actually program ;). Hiring developers is indeed very difficult. You can only tell when working with them for a few weeks.