5 ms·

Daniel Kahneman talks about an idea called substitution in his book Thinking Fast and Slow that I think really applies here. Here's the jist: When your brain

by steego 11y ago

Daniel Kahneman talks about an idea called substitution in his book Thinking Fast and Slow that I think really applies here. Here's the jist: When your brain is faced with doing a task that's going to require a lot of glucose it will look for shortcuts to save you energy. One of those shortcuts is your mind will look for an available heuristic, swap out the energy hungry analysis for the heuristic, and then signal your conscious mind that you did all the analytical hard work. https://erikreads.files.wordpress.com/2014/04/thinking-fast-and-slow-book-summary.pdf https://erikreads.files.wordpress.com/2014/04/thinking-fast-... I think the truth of that matter is, most of us (myself included), don't know how to interview people well. Interviewing is really hard. Rather than doing the hard work by researching the subject and testing ideas, most of us try to imitate successful companies much in the same way the Melanesian cargo cults imitated the construction of airfields and air traffic control towers to lure back that wonderful cargo. I suspect we like to tell ourselves that we're more analytical because our work can demand rigorous precision. More often than not, I find we developers tend to select heuristics that indirectly test a person on how similar they are to ourselves or people we aspire to be like. Then again, I'm probably making a broad generalization.

haddr 11y ago
I would also say that interviewing is hard because we have limited time to decide upon each person and what we judge is (apart from some technical qualities) not what kind of person really is but rather what we believe the person is (based on our limited human perception). And with handful of techniques a person can be really cool from at the first sight (trendy, rock star, etc.) but might be a bad choice in the long run.
- steego 11y ago
  Limited time makes it immensely difficult. However, I've found my prejudices against people not like myself have gone way past the interview. I had one intern for 2-3 months before I realized I was really lucky to have him, and it was only because someone else pointed it out. This person didn't have the right education (classes, not pedigree) nor did he project a passion for coding on his sleeve, but it turned out he was disciplined, a fast autodidact and had a very good sense of priorities. I couldn't have been more wrong about his potential. It's not just that. I think we too often discount people's ability to grow. My suspicion is we probably place too much emphasis on keeping the wrong people out and not enough on developing the people we have.
p4wnc6 11y ago
The frustrating part is that even when this is brought up and openly described and discussed explicitly in the context of working towards a better and healthier hiring practice, manager and HR types still insist on the cargo cult nonsense. It goes further than just an inefficient heuristic or biases. It becomes codified, even venerated, standard practice, and gains an air of certified approval up the corporate ladder, and eventually becomes something that, politically, you're not allowed to disagree with and typically you must even display boisterous and enthusiastic approval of it. It exacerbates the crab mentality that affects the programming world -- engineers basically start to feel that if they had to put up with stupid, inefficient interview hazing, and if it's all just going to be a political status game anyway, then they're going to drag the candidates down into the muck with them and give themselves over to designing their own hazing interview questions. The spirit becomes focused on cutting people down, and only hiring people if some gauntlet was unable to cut them down. Without ever stopping to think: what kind of human beings are you actually helping to succeed? And what kind of human beings is this process systematically harming? Nobody cares.
- cowardlydragon 11y ago
  "Good hiring" is fundamentally an altruistic ideal with all the attendant difficulties of idealistic practices in the modern workplace. This may seem ridiculous given competition for talent and cutthroat markets involved in startups and tech companies... But from the perspective of low-level hiring, where someone you hire will possibly replace or eclipse you, or where you may empower a team with more resources, or there is budgetary envy (or kill-or-be-killed cuts to funding), or easily masked discriminations aplenty (dotNET vs Java, Stanford vs Berkeley, Indian vs Pakistani, Georgia vs Alabama), conflicting management directives (we will hire the best but only pay for mediocre) or just sheer resistance to change... there are many reasons to resist transparency, hire mediocre, submarine good candidates, backstab rivals, all from obscure chaos of middle management and its wannabe Machiavellis. Management can put their smiley faces as they impose conflicting priorities and try to parse data that is fundamentally enshrouded by the multiple agendas from actors with highly variant priorities in a complex game theory all they want, but idealism is hard in cutthroat capitalism. Oh, did I mention that all candidates are lying to some degree of mendaciousness? Oh yeah, that.
dominotw 11y ago
>When your brain is faced with doing a task that's going to require a lot of glucose it will look for shortcuts to save you energy. One of those shortcuts is your mind will look for an available heuristic, swap out the energy hungry analysis for the heuristic, and then signal your conscious mind that you did all the analytical hard work. Sorry I didn't read the book but I am curious if there is any scientific evidence to back this up.
- callahad 11y ago
  Thinking Fast and Slow isn't a textbook, but Kahneman is a Nobel laureate and most of the book reports experimental findings with 30+ pages of citations in the endnotes. It's a good read.
- runamok 11y ago
  I have heard there are MRI studies that back this up. Basically people make instinctual decisions and the rational parts of their brains light up afterwards to rationalize the decision they have already made. I am on mobile but that should be searchable. Humans are very good at pattern recognition and we're optimized for it. Real calculation takes time and energy and might make us less able to survive.
  - noir_lord 11y ago
    > Real calculation takes time and energy and might make us less able to survive. Sometimes that grass moving strangely is just the wind..sometimes it's the tiger, the ancestors who sat down to have a good think about it got eaten. When I was younger and played chess my teacher drilled into me that when you find a good move that's the time to look for a better one, we instinctively play the first 'good' move we see, in fact manoeuvring your opponent into a trap by giving them an obvious 'good' move is effective against people who don't play a lot and very ineffective against people who do (and even then vastly stronger players than I ever was still fall for it occasionally).
- drags 11y ago
  There's some evidence that a related effect may not be real: http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html http://www.slate.com/articles/health_and_science/cover_story...
ThrustVectoring 11y ago
The hiring manager has a principal-agent problem going on. The manager's incentives are more around avoiding blame for bad hires than getting the best expected value for a hire at the best price.
bjt 11y ago
I think you're right about the mental shortcuts, but the main effect I see at play in the article is about misaligned incentives. It's in the organization's interest to hire the underrated, but it's not in the HR recruiter's interest to do that. If the recruiter refers a candidate who looks good on paper but turns out to be a bad hire, no one's going to blame the recruiter. But every time recruiters have the engineers interview a candidate that's not from a top school or a top company, recruiters risk losing credibility with the engineers if the candidate is rejected or ends up being a bad hire. So recruiters focus on the candidates who are the most defensible, not necessarily those who would make the company the most successful.
- taurath 11y ago
  I liken this to a government contract seller - nobody will ever get fired for choosing IBM/Microsoft/BigCo as a vendor if it fails, but you certainly can for choosing that small shop that can usually get stuff done faster. Cover Your Ass isn't a good incentive pattern when you're trying to hire engineers, which is hard enough already. At the same time, its really difficult to balance the time. My trick (and I think this is common): interview anyone that comes in on a referral within reason.
- ionforce 11y ago
  > recruiters risk losing credibility with the engineers if the candidate is rejected or ends up being a bad hire This isn't really a good way of looking at it. It should be more collaborative/problem solving rather than adversarial/blame-based. "YOU didn't get ME the RIGHT candidate."
- morgante 11y ago
  Exactly. Recruiting is ripe with principal agent problems. Recruiters don't get blamed for the undervalued candidates they failed to hire. Nor do they get fired for hiring Stanford grads. (The engineering equivalent of IBM.)
pklausler 11y ago
Unless one is interviewing for a really specific set of capabilities for a particular job, it's best to try to gauge aptitude for programming positions instead. I believe that it's possible, with the right questions, to check for basic competence in a few areas and to avoid some bad hires. I have a few small problems in my repertoire that can be solved by any good programmer in 5-10 minutes, but can't be solved in even 45 by a candidate who lacks basic understanding of how to analyze a problem, deal with basic abstractions like indirection, etc. I.e., I've given up on trying to distinguish the great candidates from the good ones, and now I just try to identify the people who just have no idea what they're doing. The error bars around the results of a 45-minute technical interview are just too damn wide, even with research and training, to do anything except try to prevent obviously bad hires.
- xenihn 11y ago
  Just curious, what are some of the problems that you use?
hinkley 11y ago
The amount of self-delusion among my fellow developers is truly breathtaking at times. Just the stuff I catch myself doing - even while this pattern is a huge pet peeve that I obsess over - is pretty bad. We need to feel smart, even though half the stuff we do is pretty damned stupid, and we keep doing it over and over again for decades before catching on. We still have watercooler conversations about issues that were identified in books that are now 30 years old. How depressing is that? Just how clever are we?
klenwell 11y ago
Kahneman also writes in the same book about his experience with interviews as a young psychologist with the Israeli army. His recommendation? A simple (but well thought out) rubric. Identify 5 or 6 qualities that are essential to the success of your team. Tailor your questions around evaluating for those. Interview a bunch of people, score them on a scale of 1 to 5 for each of those categories. Resist your gut feeling. Then hire the ones with the highest score. He acknowledges that it's crude but it was more successful than their previous process and makes as much sense as anything else I've read on the subject of hiring. Amusingly, when he got pushback from the interviewing team for not allowing them to follow their well-honed instincts, he agreed to add "Gut Feeling" as one of the 5 or 6 parameters on which candidates would be scored. That detail, for me, sums up the man's brilliance.
- ionforce 11y ago
  Does anyone else have experience with a system like this? Sounds bold. Sounds like something someone in the valley should try.
  - morgante 11y ago
    This sounds quite similar to the competency model, which basically entails listing required competencies for a position and then developing a set of specific questions to evaluate each question. Probably the only good thing I did in my first management job was to create a specific competency matrix for each position and to evaluate candidates against it. We only hired a handful of people, but I was very happy with every person I hired.
anonymousguy 11y ago
Try this over the top guide - https://github.com/prettydiff/wisdom/blob/master/Web_Interview.md https://github.com/prettydiff/wisdom/blob/master/Web_Intervi...

by steego 11y ago

haddr 11y ago

I would also say that interviewing is hard because we have limited time to decide upon each person and what we judge is (apart from some technical qualities) not what kind of person really is but rather what we believe the person is (based on our limited human perception). And with handful of techniques a person can be really cool from at the first sight (trendy, rock star, etc.) but might be a bad choice in the long run.

steego 11y ago
Limited time makes it immensely difficult. However, I've found my prejudices against people not like myself have gone way past the interview. I had one intern for 2-3 months before I realized I was really lucky to have him, and it was only because someone else pointed it out. This person didn't have the right education (classes, not pedigree) nor did he project a passion for coding on his sleeve, but it turned out he was disciplined, a fast autodidact and had a very good sense of priorities. I couldn't have been more wrong about his potential. It's not just that. I think we too often discount people's ability to grow. My suspicion is we probably place too much emphasis on keeping the wrong people out and not enough on developing the people we have.

p4wnc6 11y ago

The frustrating part is that even when this is brought up and openly described and discussed explicitly in the context of working towards a better and healthier hiring practice, manager and HR types still insist on the cargo cult nonsense. It goes further than just an inefficient heuristic or biases. It becomes codified, even venerated, standard practice, and gains an air of certified approval up the corporate ladder, and eventually becomes something that, politically, you're not allowed to disagree with and typically you must even display boisterous and enthusiastic approval of it. It exacerbates the crab mentality that affects the programming world -- engineers basically start to feel that if they had to put up with stupid, inefficient interview hazing, and if it's all just going to be a political status game anyway, then they're going to drag the candidates down into the muck with them and give themselves over to designing their own hazing interview questions. The spirit becomes focused on cutting people down, and only hiring people if some gauntlet was unable to cut them down. Without ever stopping to think: what kind of human beings are you actually helping to succeed? And what kind of human beings is this process systematically harming? Nobody cares.

cowardlydragon 11y ago
"Good hiring" is fundamentally an altruistic ideal with all the attendant difficulties of idealistic practices in the modern workplace. This may seem ridiculous given competition for talent and cutthroat markets involved in startups and tech companies... But from the perspective of low-level hiring, where someone you hire will possibly replace or eclipse you, or where you may empower a team with more resources, or there is budgetary envy (or kill-or-be-killed cuts to funding), or easily masked discriminations aplenty (dotNET vs Java, Stanford vs Berkeley, Indian vs Pakistani, Georgia vs Alabama), conflicting management directives (we will hire the best but only pay for mediocre) or just sheer resistance to change... there are many reasons to resist transparency, hire mediocre, submarine good candidates, backstab rivals, all from obscure chaos of middle management and its wannabe Machiavellis. Management can put their smiley faces as they impose conflicting priorities and try to parse data that is fundamentally enshrouded by the multiple agendas from actors with highly variant priorities in a complex game theory all they want, but idealism is hard in cutthroat capitalism. Oh, did I mention that all candidates are lying to some degree of mendaciousness? Oh yeah, that.

dominotw 11y ago

>When your brain is faced with doing a task that's going to require a lot of glucose it will look for shortcuts to save you energy. One of those shortcuts is your mind will look for an available heuristic, swap out the energy hungry analysis for the heuristic, and then signal your conscious mind that you did all the analytical hard work. Sorry I didn't read the book but I am curious if there is any scientific evidence to back this up.

callahad 11y ago
Thinking Fast and Slow isn't a textbook, but Kahneman is a Nobel laureate and most of the book reports experimental findings with 30+ pages of citations in the endnotes. It's a good read.
runamok 11y ago
I have heard there are MRI studies that back this up. Basically people make instinctual decisions and the rational parts of their brains light up afterwards to rationalize the decision they have already made. I am on mobile but that should be searchable. Humans are very good at pattern recognition and we're optimized for it. Real calculation takes time and energy and might make us less able to survive.
- noir_lord 11y ago
  > Real calculation takes time and energy and might make us less able to survive. Sometimes that grass moving strangely is just the wind..sometimes it's the tiger, the ancestors who sat down to have a good think about it got eaten. When I was younger and played chess my teacher drilled into me that when you find a good move that's the time to look for a better one, we instinctively play the first 'good' move we see, in fact manoeuvring your opponent into a trap by giving them an obvious 'good' move is effective against people who don't play a lot and very ineffective against people who do (and even then vastly stronger players than I ever was still fall for it occasionally).
drags 11y ago
There's some evidence that a related effect may not be real: http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html http://www.slate.com/articles/health_and_science/cover_story...

ThrustVectoring 11y ago

The hiring manager has a principal-agent problem going on. The manager's incentives are more around avoiding blame for bad hires than getting the best expected value for a hire at the best price.

bjt 11y ago

I think you're right about the mental shortcuts, but the main effect I see at play in the article is about misaligned incentives. It's in the organization's interest to hire the underrated, but it's not in the HR recruiter's interest to do that. If the recruiter refers a candidate who looks good on paper but turns out to be a bad hire, no one's going to blame the recruiter. But every time recruiters have the engineers interview a candidate that's not from a top school or a top company, recruiters risk losing credibility with the engineers if the candidate is rejected or ends up being a bad hire. So recruiters focus on the candidates who are the most defensible, not necessarily those who would make the company the most successful.

taurath 11y ago
I liken this to a government contract seller - nobody will ever get fired for choosing IBM/Microsoft/BigCo as a vendor if it fails, but you certainly can for choosing that small shop that can usually get stuff done faster. Cover Your Ass isn't a good incentive pattern when you're trying to hire engineers, which is hard enough already. At the same time, its really difficult to balance the time. My trick (and I think this is common): interview anyone that comes in on a referral within reason.
ionforce 11y ago
> recruiters risk losing credibility with the engineers if the candidate is rejected or ends up being a bad hire This isn't really a good way of looking at it. It should be more collaborative/problem solving rather than adversarial/blame-based. "YOU didn't get ME the RIGHT candidate."
morgante 11y ago
Exactly. Recruiting is ripe with principal agent problems. Recruiters don't get blamed for the undervalued candidates they failed to hire. Nor do they get fired for hiring Stanford grads. (The engineering equivalent of IBM.)

pklausler 11y ago

Unless one is interviewing for a really specific set of capabilities for a particular job, it's best to try to gauge aptitude for programming positions instead. I believe that it's possible, with the right questions, to check for basic competence in a few areas and to avoid some bad hires. I have a few small problems in my repertoire that can be solved by any good programmer in 5-10 minutes, but can't be solved in even 45 by a candidate who lacks basic understanding of how to analyze a problem, deal with basic abstractions like indirection, etc. I.e., I've given up on trying to distinguish the great candidates from the good ones, and now I just try to identify the people who just have no idea what they're doing. The error bars around the results of a 45-minute technical interview are just too damn wide, even with research and training, to do anything except try to prevent obviously bad hires.

xenihn 11y ago
Just curious, what are some of the problems that you use?

hinkley 11y ago

The amount of self-delusion among my fellow developers is truly breathtaking at times. Just the stuff I catch myself doing - even while this pattern is a huge pet peeve that I obsess over - is pretty bad. We need to feel smart, even though half the stuff we do is pretty damned stupid, and we keep doing it over and over again for decades before catching on. We still have watercooler conversations about issues that were identified in books that are now 30 years old. How depressing is that? Just how clever are we?

klenwell 11y ago

Kahneman also writes in the same book about his experience with interviews as a young psychologist with the Israeli army. His recommendation? A simple (but well thought out) rubric. Identify 5 or 6 qualities that are essential to the success of your team. Tailor your questions around evaluating for those. Interview a bunch of people, score them on a scale of 1 to 5 for each of those categories. Resist your gut feeling. Then hire the ones with the highest score. He acknowledges that it's crude but it was more successful than their previous process and makes as much sense as anything else I've read on the subject of hiring. Amusingly, when he got pushback from the interviewing team for not allowing them to follow their well-honed instincts, he agreed to add "Gut Feeling" as one of the 5 or 6 parameters on which candidates would be scored. That detail, for me, sums up the man's brilliance.

ionforce 11y ago
Does anyone else have experience with a system like this? Sounds bold. Sounds like something someone in the valley should try.
- morgante 11y ago
  This sounds quite similar to the competency model, which basically entails listing required competencies for a position and then developing a set of specific questions to evaluate each question. Probably the only good thing I did in my first management job was to create a specific competency matrix for each position and to evaluate candidates against it. We only hired a handful of people, but I was very happy with every person I hired.

anonymousguy 11y ago

Try this over the top guide - https://github.com/prettydiff/wisdom/blob/master/Web_Interview.md https://github.com/prettydiff/wisdom/blob/master/Web_Intervi...