5 ms·

I think the Dunning-Kruger thing is a nice idea but it's immediately ruined once you become aware of it, like a Heisenberg uncertainty sort of thing. I used to

I think the Dunning-Kruger thing is a nice idea but it's immediately ruined once you become aware of it, like a Heisenberg uncertainty sort of thing. I used to think I was crap, but Dunning-Kruger, so I must be great. I used to think I was great, but Dunning-Kruger, so I must be crap. It just seems a bit trite to base any real world conclusions or behaviour off of. I agree that the interview process is broken. The reason I wouldn't apply to Google/Facebook/Fog Creek is personal and nothing to do with their interview process.

wpietri 12y ago
> I think the Dunning-Kruger thing is a nice idea but it's immediately ruined once you become aware of it, like a Heisenberg uncertainty sort of thing. Not at all! You're being too binary with it. Dunning-Kruger is about the relationship between discernment and capability. In particular, it's saying discernment drives capability. A great way of applying it is this Ira Glass quote: http://www.goodreads.com/quotes/309485-nobody-tells-this-to-people-who-are-beginners-i-wish http://www.goodreads.com/quotes/309485-nobody-tells-this-to-... Basically, the way I apply it is to say that if I can tell that my work isn't as good as I want it to be, that's ok: I have to use that as fuel to improve. And if I'm dealing with something where my work looks perfect to me, it's time to up my discernment. In hiring, the way I apply it is to avoid listening much to self-assessment and related measures of competence (e.g., confidence). Instead, I go for demonstrations of competence, like short essay answers instead of cover letters, or pair programming interviews [1] over "tell me about a time when" discussions. [1] https://www.quora.com/What-are-the-best-programming-interview-questions-youve-ever-asked-or-been-asked/answer/William-Pietri?share=1 https://www.quora.com/What-are-the-best-programming-intervie...
- throwawayaway 12y ago
  I like that Ira Glass quote, maybe some day I'll get there. I think you are just positively improving your work, I don't think what you are saying is an application of Dunning-Kruger. Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second? Dunning-Kruger says both assessments are likely to be wrong, and in both cases you are striving to improve - so Dunning-Kruger has little to do with it. In fact you could go further in a "worse is better" sense and say your striving to improve one thing is letting down something else you could be paying attention to. Dunning did an AMA on reddit a few months back: http://www.reddit.com/r/science/comments/2m6d68 http://www.reddit.com/r/science/comments/2m6d68 Your hiring approach does sound very common sense and I am sure people did as you do long before Dunning Kruger was a thing.
  - wpietri 12y ago
    It's about how one strives to improve. Have you read the original paper? http://psych.colorado.edu/~vanboven/teaching/p7536_heurbias/p7536_readings/kruger_dunning.pdf http://psych.colorado.edu/~vanboven/teaching/p7536_heurbias/... Incompetent people think they're doing fine because they can't tell the difference between good and bad work. Highly competent people, readily seeing their own flaws, think they don't do as well as they are doing. (At least until you have them grade a bunch of other work, at which point they are better calibrated.) Competent people are competent because their metacognitive ability, their ability to tell good work from bad, drives them forward. So basically, if I already can see the problems in my work, then my metacognitive ability is sufficient to drive improvement. When I improve enough that my work looks fine to me, further improvement is prevented because I literally can't tell whether changes I make are better or worse. There, the path to improvement is to improve metacognition. As to the latter, my approach may be common sense, but it is not in fact common. Last I was interviewing developers, most people had never done a pair programming interview before. The default is talk-talk interviews. I'm sure some people did it the other way, of course: science frequently trails practice. But the Dunning-Kruger paper in specific is what convinced me that self-assessments were at best highly unreliable and at worst thoroughly misleading.
    - throwawayaway 12y ago
      haven't read it thanks. are you saying the paper is about how one strives to improve? - because it's not titled that. the title is pretty clear. so is the abstract. i think we agree that self assessments aren't worth much. i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency.
      - wpietri 12y ago
        > haven't read it thanks. are you saying the paper is about how one strives to improve? - because it's not titled that. the title is pretty clear. so is the abstract. I am not. You didn't see how one could usefully apply their work. I'm explaining how I have applied it. You asked "Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second?" When I say, "It's about how one strives to improve," I am answering your question. > i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency. Not at all. If for X I am a person of low ability, then I am likely to perceive myself as doing pretty well at X because I cannot see the flaws in my work. So if I'm at a state where my work looks good to me, the thing to do is, as I said, to improve my discernment until I can see flaws in my work. Then I can work to improve, increasing my competency. This strategy works no matter what you relative competence is. And note that relative competence is distinct from absolute competence. Even people in the top quartile of a given sample can keep improving, and I believe Glass's quote helps explain why: their taste is still ahead of their skill, which if they keep practicing yields increased absolute competence.

wpietri 12y ago

> I think the Dunning-Kruger thing is a nice idea but it's immediately ruined once you become aware of it, like a Heisenberg uncertainty sort of thing. Not at all! You're being too binary with it. Dunning-Kruger is about the relationship between discernment and capability. In particular, it's saying discernment drives capability. A great way of applying it is this Ira Glass quote: http://www.goodreads.com/quotes/309485-nobody-tells-this-to-people-who-are-beginners-i-wish http://www.goodreads.com/quotes/309485-nobody-tells-this-to-... Basically, the way I apply it is to say that if I can tell that my work isn't as good as I want it to be, that's ok: I have to use that as fuel to improve. And if I'm dealing with something where my work looks perfect to me, it's time to up my discernment. In hiring, the way I apply it is to avoid listening much to self-assessment and related measures of competence (e.g., confidence). Instead, I go for demonstrations of competence, like short essay answers instead of cover letters, or pair programming interviews [1] over "tell me about a time when" discussions. [1] https://www.quora.com/What-are-the-best-programming-interview-questions-youve-ever-asked-or-been-asked/answer/William-Pietri?share=1 https://www.quora.com/What-are-the-best-programming-intervie...

throwawayaway 12y ago
I like that Ira Glass quote, maybe some day I'll get there. I think you are just positively improving your work, I don't think what you are saying is an application of Dunning-Kruger. Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second? Dunning-Kruger says both assessments are likely to be wrong, and in both cases you are striving to improve - so Dunning-Kruger has little to do with it. In fact you could go further in a "worse is better" sense and say your striving to improve one thing is letting down something else you could be paying attention to. Dunning did an AMA on reddit a few months back: http://www.reddit.com/r/science/comments/2m6d68 http://www.reddit.com/r/science/comments/2m6d68 Your hiring approach does sound very common sense and I am sure people did as you do long before Dunning Kruger was a thing.
- wpietri 12y ago
  It's about how one strives to improve. Have you read the original paper? http://psych.colorado.edu/~vanboven/teaching/p7536_heurbias/p7536_readings/kruger_dunning.pdf http://psych.colorado.edu/~vanboven/teaching/p7536_heurbias/... Incompetent people think they're doing fine because they can't tell the difference between good and bad work. Highly competent people, readily seeing their own flaws, think they don't do as well as they are doing. (At least until you have them grade a bunch of other work, at which point they are better calibrated.) Competent people are competent because their metacognitive ability, their ability to tell good work from bad, drives them forward. So basically, if I already can see the problems in my work, then my metacognitive ability is sufficient to drive improvement. When I improve enough that my work looks fine to me, further improvement is prevented because I literally can't tell whether changes I make are better or worse. There, the path to improvement is to improve metacognition. As to the latter, my approach may be common sense, but it is not in fact common. Last I was interviewing developers, most people had never done a pair programming interview before. The default is talk-talk interviews. I'm sure some people did it the other way, of course: science frequently trails practice. But the Dunning-Kruger paper in specific is what convinced me that self-assessments were at best highly unreliable and at worst thoroughly misleading.
  - throwawayaway 12y ago
    haven't read it thanks. are you saying the paper is about how one strives to improve? - because it's not titled that. the title is pretty clear. so is the abstract. i think we agree that self assessments aren't worth much. i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency.
    - wpietri 12y ago
      > haven't read it thanks. are you saying the paper is about how one strives to improve? - because it's not titled that. the title is pretty clear. so is the abstract. I am not. You didn't see how one could usefully apply their work. I'm explaining how I have applied it. You asked "Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second?" When I say, "It's about how one strives to improve," I am answering your question. > i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency. Not at all. If for X I am a person of low ability, then I am likely to perceive myself as doing pretty well at X because I cannot see the flaws in my work. So if I'm at a state where my work looks good to me, the thing to do is, as I said, to improve my discernment until I can see flaws in my work. Then I can work to improve, increasing my competency. This strategy works no matter what you relative competence is. And note that relative competence is distinct from absolute competence. Even people in the top quartile of a given sample can keep improving, and I believe Glass's quote helps explain why: their taste is still ahead of their skill, which if they keep practicing yields increased absolute competence.