7 ms·

The way I interpret DKE is that low level people don’t (yet) know what they don’t know. One thing that I think often gets skipped over in discussions on Impost

The way I interpret DKE is that low level people don’t (yet) know what they don’t know. One thing that I think often gets skipped over in discussions on Imposter Syndrome is that imposters DO exist (that’s kind of why we use interviews when hiring people).

karmakaze 6y ago
Yes, the wikipedia quote seems incomplete "The miscalibration of the incompetent stems from an error about the self, [...]" The source of miscalibration is from the estimation of the subject area and the assessment of how much of that has been learned. I suspect that the error in the estimation of the subject space is the much larger source of error.
xyzelement 6y ago
>> The way I interpret DKE is that low level people don’t (yet) know what they don’t know. Exactly, it's like people who think the company CEO is a useless idiot because they don't see him writing React code or whatever their limited definition of "adding value" is. Another example I see on HN - whenever a hiring thread comes up, people chime in how stupid whiteboard interviews are, and surely the poster would be a super-star at a FAANG if only they weren't gate-kept by these dumb interviews. Isn't it more likely that these super-successful companies use these interviews because they end up hiring the people they want to hire, and it's you, the poster, lacking a clue as to what that looks like? Basically the heuristic is, if something/someone is successful and it seems stupid to you, start by assuming it's you who likely has something to learn.
- rozularen 6y ago
  Your last sentence made me think about Hanlon's Razor[0] [0] https://en.m.wikipedia.org/wiki/Hanlon%27s_razor https://en.m.wikipedia.org/wiki/Hanlon%27s_razor
- Moodles 6y ago
  > Isn't it more likely that these super-successful companies use these interviews because they end up hiring the people they want to hire...? I’m not sure it’s that these companies only hire the people they want to hire. It’s probably that highly successful companies know their interview methods don’t work 100% of the time, but they’re willing to tolerate the false negatives of not hiring a few good people if it means all those they do hire meet some minimum standards. And their current system is the best they know how to do. We arrogantly post on Hacker News: “Oh look, this interview method sucks! It will miss out on this great candidate!”. Ok then, make your own trillion dollar company with a better interview method that filters out the hordes of bullshit artists while simultaneously never missing out on the best people. Filtering out the bullshit artists is more important than hiring every single productive person. In fact, if you hire too many bullshit artists, all the good people will get fed up and leave anyway.
  - hn_throwaway_99 6y ago
    Joel Spolsky has a very good (now old) blog post about this: https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/ https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... Basically the argument is that it is much better to miss out on a good person than hire a mediocre/bad person.
  - jfim 6y ago
    It's still a frustrating experience for candidates though. If you were to split the candidates into three buckets, with the first bucket being people who would never pass a FAANG interview (can't code their way out of a wet paper bag), people who can pass a FAANG interview but not reliably, and people who can reliably pass FAANG interviews (PhD in computer science or just leetcode grinder), most of the people in the first two buckets would complain about the interview process. As someone in the second bucket, the process feels rather random (eg. passed the Facebook L6 interview but failed the Google L4 one somehow, passed the onsite at half a dozen other companies), which is mostly because it is. Maybe some interviewer really wanted to see a topological sort algorithm implemented in 20 minutes and there's not enough time in the interview to derive it from first principles, or maybe an interviewer felt grumpy that day, and then that's it, better luck next time. Combined with the policy of not telling candidates what to improve on, it's a pretty frustrating experience. Yes, there are reasons for the company to behave that way, but it's still unpleasant to be on the receiving end of it.
- hn_throwaway_99 6y ago
  Glad you brought up the hiring threads on HN. Recently I've felt so many of these threads have devolved into "these stupid companies using whiteboard hiring are all a bunch of idiots!" that I usually stop commenting. What's most illuminating about those kinds of comments is they rarely even try to view thing from the hiring company's perspective. Having been in that boat, you'll find: 1. The cost of a mishire is huge. And, even worse than hiring someone who is downright awful is hiring someone who is just plain mediocre, or slow. I mean, I've hired diligent, hard-working, otherwise smart people who could only get stuff out the door at about a third the rate of other devs. Having a performance discussion with those folks can be extremely difficult, because oftentimes there is little they can realistically do to go faster. 2. Experience can have little correlation with proficiency. I've interviewed people who were "senior engineers" and "architects" at major corporation who couldn't code FizBuzz, moreover they could barely code a syntactically correct function in their chosen language. 3. Many companies want to diversify their employee base by going outside just friends/referrals and well known college programs, but to "take a chance" on other folks means they need to get a strong signal during the interview process. I'm not saying there aren't problems in hiring, but given how there is a giant economic incentive to make hiring as accurate and efficient as possible it's at least valuable to attempt to understand why a company might interview the way it does.
  - doctorbaum 6y ago
    But how whiteboard interviews help (1)? I see them as unconnected and would prefer for that (for example) giving a sample assignment for 2 hours on/off-site
    - xyzelement 6y ago
      I hope this insight helps you and others because it's key: What does an off-site sample assignment tell the company? That you can write code when given a spec. That's great, but that's insufficient for what they are hiring for. What it doesn't tell them: - Can you elicit clarity on a vaguely stated problem? These are conversations that top engineers have with their business counterparts and other technologists constantly. - Can you iterate with another person on a solution. The whiteboarding exercise is a dialogue, and while contrived, problems are often solved in joint manner like that. - How do you do under pressure? Something is going wrong and it's affecting millions of users. Can you carry your weight when the team is fire fighting? Obviously the whiteboard isn't perfectly correlated with the above, but it's as good an indicator as I can think of. Saying "just let me write something off-line" means people are oblivious to these other key attributes. FAANG devs aren't earning 300K a year for just being able to write code on a spec.
- 13415 6y ago
  Although I agree in general, the example you've chosen might not be the best one. This might be one of (numerous) areas in business where success is not measured correctly and the methods used are never really validated. Do human resources departments actually evaluate their hiring criteria and the tests they use in experimental designs, e.g. by hiring two sufficiently large groups A and B based on different interview methods and checking a year later how the groups performed relative to each other? Bear in mind that there are plenty of people who believe that they're good at doing something just because they have been doing it for a long time. Without an independent method for confirming such claims they are practically useless.
  - dkarl 6y ago
    In theory, I like your idea, but in practice, I think corporations large enough to run experiments like that are not able to objectively evaluate anything they do, at least not officially. It's too political. I've been in dozens of interviews as an interviewer and worked with at least a dozen people after seeing how they performed in interviews, so that's what I go on.
    - xyzelement 6y ago
      The big software firm I worked at (not FAANG but we competed with them for talent) absolutely tweaked its interview processes based on long run outcomes. FAANGS evolve their process over time too. One clear example - at FAANGS even if the manager likes you, it doesn't matter. You have to be evaluated by a committee of people who're indifferent to your manager, aren't under the gun to fill a slot and are less likely to be swayed by a one-time positive conversation. Do you think this process came out of nowhere? That's just one example of them settling on something that worked, I am sure having previously tried other less structured approaches and tracking the results.
  - xyzelement 6y ago
    There's two things at play here. First, candidates who complain about the whiteboard interviews uniformly complain about it "not being a good representation of how I can write code" which betrays a lack of understanding of the other skills besides writing code that these firms care about. People who understand more can connect the interview type to the other relevant attributes, so it's already clear who's more right. Second, yes, companies do a shitload of triangulation on recruiting performance including looking many years out.
- Gibbon1 6y ago
  > Isn't it more likely that these super-successful companies use these interviews because they end up hiring the people they want to hire, and it's you, the poster, lacking a clue as to what that looks like? No it's more likely that the people staffing FAANG companies absolutely suck at interviewing and vetting.
JamesBarney 6y ago
The whole article was about how that's probably not the case and it's better explained by other factors.
deleted 6y ago
[deleted]
dragonwriter 6y ago
> The way I interpret DKE is that low level people don’t (yet) know what they don’t know. That's obviously a part of it, but if that was all of it, you would expect accuracy of assessment of relative ability to consistently get better with relative ability. But, in fact, the result is basically “everyone seems themselves as closer to the 70th percentile than they actually are”. Everyone focusses on the low end of DKE to use it as a low-brow dismissal, but the fact that people at the top of the distribution tend to underestimate their relative ability is also interesting. > imposters DO exist (that’s kind of why we use interviews when hiring people). Yes, the presence of imposters in hiring and hiring-policy-setting positions is why we use assessment tools that don't work very well to evaluate candidates.

karmakaze 6y ago

Yes, the wikipedia quote seems incomplete "The miscalibration of the incompetent stems from an error about the self, [...]" The source of miscalibration is from the estimation of the subject area and the assessment of how much of that has been learned. I suspect that the error in the estimation of the subject space is the much larger source of error.

xyzelement 6y ago

>> The way I interpret DKE is that low level people don’t (yet) know what they don’t know. Exactly, it's like people who think the company CEO is a useless idiot because they don't see him writing React code or whatever their limited definition of "adding value" is. Another example I see on HN - whenever a hiring thread comes up, people chime in how stupid whiteboard interviews are, and surely the poster would be a super-star at a FAANG if only they weren't gate-kept by these dumb interviews. Isn't it more likely that these super-successful companies use these interviews because they end up hiring the people they want to hire, and it's you, the poster, lacking a clue as to what that looks like? Basically the heuristic is, if something/someone is successful and it seems stupid to you, start by assuming it's you who likely has something to learn.

rozularen 6y ago
Your last sentence made me think about Hanlon's Razor[0] [0] https://en.m.wikipedia.org/wiki/Hanlon%27s_razor https://en.m.wikipedia.org/wiki/Hanlon%27s_razor
Moodles 6y ago
> Isn't it more likely that these super-successful companies use these interviews because they end up hiring the people they want to hire...? I’m not sure it’s that these companies only hire the people they want to hire. It’s probably that highly successful companies know their interview methods don’t work 100% of the time, but they’re willing to tolerate the false negatives of not hiring a few good people if it means all those they do hire meet some minimum standards. And their current system is the best they know how to do. We arrogantly post on Hacker News: “Oh look, this interview method sucks! It will miss out on this great candidate!”. Ok then, make your own trillion dollar company with a better interview method that filters out the hordes of bullshit artists while simultaneously never missing out on the best people. Filtering out the bullshit artists is more important than hiring every single productive person. In fact, if you hire too many bullshit artists, all the good people will get fed up and leave anyway.
- hn_throwaway_99 6y ago
  Joel Spolsky has a very good (now old) blog post about this: https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/ https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... Basically the argument is that it is much better to miss out on a good person than hire a mediocre/bad person.
- jfim 6y ago
  It's still a frustrating experience for candidates though. If you were to split the candidates into three buckets, with the first bucket being people who would never pass a FAANG interview (can't code their way out of a wet paper bag), people who can pass a FAANG interview but not reliably, and people who can reliably pass FAANG interviews (PhD in computer science or just leetcode grinder), most of the people in the first two buckets would complain about the interview process. As someone in the second bucket, the process feels rather random (eg. passed the Facebook L6 interview but failed the Google L4 one somehow, passed the onsite at half a dozen other companies), which is mostly because it is. Maybe some interviewer really wanted to see a topological sort algorithm implemented in 20 minutes and there's not enough time in the interview to derive it from first principles, or maybe an interviewer felt grumpy that day, and then that's it, better luck next time. Combined with the policy of not telling candidates what to improve on, it's a pretty frustrating experience. Yes, there are reasons for the company to behave that way, but it's still unpleasant to be on the receiving end of it.
hn_throwaway_99 6y ago
Glad you brought up the hiring threads on HN. Recently I've felt so many of these threads have devolved into "these stupid companies using whiteboard hiring are all a bunch of idiots!" that I usually stop commenting. What's most illuminating about those kinds of comments is they rarely even try to view thing from the hiring company's perspective. Having been in that boat, you'll find: 1. The cost of a mishire is huge. And, even worse than hiring someone who is downright awful is hiring someone who is just plain mediocre, or slow. I mean, I've hired diligent, hard-working, otherwise smart people who could only get stuff out the door at about a third the rate of other devs. Having a performance discussion with those folks can be extremely difficult, because oftentimes there is little they can realistically do to go faster. 2. Experience can have little correlation with proficiency. I've interviewed people who were "senior engineers" and "architects" at major corporation who couldn't code FizBuzz, moreover they could barely code a syntactically correct function in their chosen language. 3. Many companies want to diversify their employee base by going outside just friends/referrals and well known college programs, but to "take a chance" on other folks means they need to get a strong signal during the interview process. I'm not saying there aren't problems in hiring, but given how there is a giant economic incentive to make hiring as accurate and efficient as possible it's at least valuable to attempt to understand why a company might interview the way it does.
- doctorbaum 6y ago
  But how whiteboard interviews help (1)? I see them as unconnected and would prefer for that (for example) giving a sample assignment for 2 hours on/off-site
  - xyzelement 6y ago
    I hope this insight helps you and others because it's key: What does an off-site sample assignment tell the company? That you can write code when given a spec. That's great, but that's insufficient for what they are hiring for. What it doesn't tell them: - Can you elicit clarity on a vaguely stated problem? These are conversations that top engineers have with their business counterparts and other technologists constantly. - Can you iterate with another person on a solution. The whiteboarding exercise is a dialogue, and while contrived, problems are often solved in joint manner like that. - How do you do under pressure? Something is going wrong and it's affecting millions of users. Can you carry your weight when the team is fire fighting? Obviously the whiteboard isn't perfectly correlated with the above, but it's as good an indicator as I can think of. Saying "just let me write something off-line" means people are oblivious to these other key attributes. FAANG devs aren't earning 300K a year for just being able to write code on a spec.
13415 6y ago
Although I agree in general, the example you've chosen might not be the best one. This might be one of (numerous) areas in business where success is not measured correctly and the methods used are never really validated. Do human resources departments actually evaluate their hiring criteria and the tests they use in experimental designs, e.g. by hiring two sufficiently large groups A and B based on different interview methods and checking a year later how the groups performed relative to each other? Bear in mind that there are plenty of people who believe that they're good at doing something just because they have been doing it for a long time. Without an independent method for confirming such claims they are practically useless.
- dkarl 6y ago
  In theory, I like your idea, but in practice, I think corporations large enough to run experiments like that are not able to objectively evaluate anything they do, at least not officially. It's too political. I've been in dozens of interviews as an interviewer and worked with at least a dozen people after seeing how they performed in interviews, so that's what I go on.
  - xyzelement 6y ago
    The big software firm I worked at (not FAANG but we competed with them for talent) absolutely tweaked its interview processes based on long run outcomes. FAANGS evolve their process over time too. One clear example - at FAANGS even if the manager likes you, it doesn't matter. You have to be evaluated by a committee of people who're indifferent to your manager, aren't under the gun to fill a slot and are less likely to be swayed by a one-time positive conversation. Do you think this process came out of nowhere? That's just one example of them settling on something that worked, I am sure having previously tried other less structured approaches and tracking the results.
- xyzelement 6y ago
  There's two things at play here. First, candidates who complain about the whiteboard interviews uniformly complain about it "not being a good representation of how I can write code" which betrays a lack of understanding of the other skills besides writing code that these firms care about. People who understand more can connect the interview type to the other relevant attributes, so it's already clear who's more right. Second, yes, companies do a shitload of triangulation on recruiting performance including looking many years out.
Gibbon1 6y ago
> Isn't it more likely that these super-successful companies use these interviews because they end up hiring the people they want to hire, and it's you, the poster, lacking a clue as to what that looks like? No it's more likely that the people staffing FAANG companies absolutely suck at interviewing and vetting.

JamesBarney 6y ago

The whole article was about how that's probably not the case and it's better explained by other factors.

deleted 6y ago

[deleted]

dragonwriter 6y ago

> The way I interpret DKE is that low level people don’t (yet) know what they don’t know. That's obviously a part of it, but if that was all of it, you would expect accuracy of assessment of relative ability to consistently get better with relative ability. But, in fact, the result is basically “everyone seems themselves as closer to the 70th percentile than they actually are”. Everyone focusses on the low end of DKE to use it as a low-brow dismissal, but the fact that people at the top of the distribution tend to underestimate their relative ability is also interesting. > imposters DO exist (that’s kind of why we use interviews when hiring people). Yes, the presence of imposters in hiring and hiring-policy-setting positions is why we use assessment tools that don't work very well to evaluate candidates.