4 ms·
The Dunning-Kruger Effect Is Probably Not Real
- mistercow 6y agoAt the very least, I wish people would look at those graphs, and realize that the popular “mount stupid” formulation is basically fan fiction.
- ncmncm 6y agoAs I say, "Citations of Dunning-Kruger are usually examples of it."
- takeiteasyy 6y agoGave me a good chuckle
- Rochus 6y agofunny recursion (if intended)
- draw_down 6y agoThat’s right
- addicted 6y agoThe Dunning Kruger might be a mirage but this article doesn’t seem to show that. It shows 2 charts, one that was used by Dunning Kruger and the other generated by random noise. Both look similar, but in the random noise chart the 2 lines intersect midway along the x-axis, where the 2nd and 3quartiles meet, while the dinning Kruger chart has the line intersecting where the 3rd and 4th quartiles meet. Isn’t that a significant difference between the 2 results?
- NiceWayToDoIT 6y agoIn my experience Dunning-Kruger 100% is there but in the following form: "knowledgeable people doubt, and less people know they are more confident." It is even amazing sometimes, how confidently people without any knowledge talk and give opinion about subjects that need high level of knowledge, just to even begin to ponder about it. And again, it can be my bias.
- elmo2you 6y agoI don't know why your response got down voted. Maybe some rather specific people got their panties in a twist. Either way, what you said is something Bertrand Russel (certainly not best known for being a dumb man) once worded like this: "The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt." .. and just for good measure, here is another one of his quotes, which is hardly well know to those who would probably be better off if they did: "Our great democracies still tend to think that a stupid man is more likely to be honest than a clever man, and our politicians take advantage of this prejudice by pretending to be even more stupid than nature made them." In summary: ignorant people do appear to often be more confident than they deserve to be. Whether that is actually because they are truly unaware of their ignorance, that could certainly be a good question to ask oneself. For there are plenty who deliberately abuse both faked ignorance and confidence just to bamboozle and deceive others.
- NiceWayToDoIT 6y agoSomeone said "if you wouldn't attack yourself in street fight don't do it verbally", but for the sake of good British joke I will say following: So, down-voting my answer is confirming my statement. I do not know much about subject but I confidently claimed 100% effect, therefore confirming what I said :) It is like that brain melting sentence "If I say that I am lying, am I lying?" If I claim that Dunning-Kruger Effect is 100% real is it really real? :) Marry Christmas and Happy Holidays everyone!
- tyu2 6y ago> I do not know much about subject but I confidently claimed 100% effect, therefore confirming what I said :) Well, that's not a Dunning-Kruger Effect either, but maybe a confirmation bias mixed with survivorship bias.
- NiceWayToDoIT 6y agoPffffffffff *amn it ... I can not be right about being wrong in a proper way ...
- echelon 6y agoI can't go a week without seeing people mention Dunning-Kruger on Reddit. It's become such a pop-cultural zeitgeist term that it's lost any ties to the original study. It's frequently abused as a derogatory adjective to describe people while claiming the superior argument, often without providing evidence. I don't know what to do about it, but its use by lay people bothers me. It's ironically Dunning-Kruger.
- seibelj 6y agoThis is a popular opinion on HN, but I'll state again that reddit is basically unusable. I still check a few subreddits using old.reddit.com but the mobile app they constantly shove down your throat is bloated trash and the redesign is garbage. It's inevitable that something better comes along.
- tyu2 6y agoYou can try to explain that the effect doesn't actually apply to any pointing out of the effect in someone's comment, maybe even cite the study.
- richk449 6y agoAgreed. And its use in discussions unfortunately adds no value. Imagine that two people are arguing about something. You walk up, interrupt them, and state that one of them is wrong but because the wrong person is stupid, they don't realize they are wrong. Will your comment change anything? Of course not, both people arguing will just assume that the other one is the stupid person.
- Bakary 6y agoI can't help but notice that your comment ironically continues the recursion chain, and that mine now does as well.
- lr4444lr 6y agoPersonally speaking, it just struck me as odd that - outside of some rare exceptions - that someone could be so dumb that he'd actually be ignorant of his ineptitude. This is different than misrepresenting your ability. Lots of people do that for lots of reasons, but for someone to be so sheerly ignorant as is often given in the case examples struck me as someone who will sooner rather than later learn painful lessons about the truth of his own shortcomings.
- emerged 6y agoThe people I know who fall into this category construct an elaborate world view where they are the constant victim of all the idiots and jerks of the world. Everything that goes wrong is someone else’s fault, not their own ignorance and bad choices.
- MeinBlutIstBlau 6y agoSo half of reddit?
- mhh__ 6y agoMaybe the HN delusions of grandeur effect is real after all (...I like both, merry Christmas)
- MeinBlutIstBlau 6y agoI'm not saying reddit doesnt have any uses. But /r/popular is not what it once was. Like imagine if HN was covered in memes and low quality content.
- iratewizard 6y agoYou're grossly underestimating
- Caylio 6y agoIts the DK effect in action.
- elipsey 6y agoI'm sure the Dunning-Kruger effect is real, and I'm much better than average at being able to tell!
- Isinlor 6y agoDoes this article actually contains explanation of what was wrong with the original study? Maybe I don't know how to read with understanding, but to me this article makes only two points: 1. The effect was about all people, not just dumb people. 2. The same graph can be obtained "by random". Point 1. makes sense, but 2. is left unexplained. There is a lot of words, but no content. Almost as if GPT-3 was writing it :) .
- Will_Do 6y agoYea I agree. It all relies on (2) but they do a very poor job explaining what exactly the simulation is. If there is a paper by McNight, it isn't cited and I can't find it. Can't find it on his webpage[0][1] or google scholar. This is a non-trivial simulation so the details are really important! Shame they are nowhere to be found. [0]: https://ibi.gmu.edu/faculty-directory/patrick-mcknight/ https://ibi.gmu.edu/faculty-directory/patrick-mcknight/ [1]:https://scholar.google.com/citations?hl=en&user=sH44LC4AAAAJ&view_op=list_works&sortby=pubdate https://scholar.google.com/citations?hl=en&user=sH44LC4AAAAJ...
- SoSoRoCoCo 6y agoExactly! Why does the author not tell us how the McKnight model works? This is the only substantial claim the author makes, yet the result is delivered in two measly sentences. I feel snookered.
- thethirdone 6y agoThe other paper by Ackermann also has a simulated graph, but I have yet to find the simulation model.
- lrossi 6y agoOn top of that, the graphs are only barely similar, and I would not call them the same. There is a clear difference in magnitude.
- deleted 6y ago[deleted]
- 6y ago
- kkwteh 6y agoThe supposed smoking gun in the article is that if we plot random values for perceived and actual ability we get a chart that looks a lot like the chart in the original study. I don't see how you make the leap from there to saying that the Dunning-Kruger effect is probably not real. The headline is totally overblown if you ask me.
- jonnycomputer 6y agoI guess they could just be more epistemically modest and say that the data doesn't support the existence of the Dunning-Kruger effect, but doesn't make it not real.
- jonnycomputer 6y agoMy goodness, two downvotes for distinguishing between a failure to show something exists and proof that it doesn't exist.
- scotty79 6y agoIf you claim that you have meaningful pattern in the data, but someone replicates your pattern with meaningless data then it's perfectly valid to argue that your pattern doesn't come from the data but rather from how you chose to process the data.
- ilitirit 6y ago> The darling of those who wish to explain why incompetent people don’t know they’re unskilled, the Dunning-Kruger effect may actually just be a data artefact. This doesn't strike me as true. Or rather, I don't think that it's a significant observation that some people may want to try to explain why some people are incompetent. In my experience, people don't try to "explain" incompetence in general. They will rather talk about specifics - e.g. "Some people believe this because of the misconception that X implies Y". Even if their own rationale is incorrect. This is very different to saying that "They are stupid because they don't know that they are stupid". Yeah it may come up as a flippant comment about an individual (or "fringe" group like the Flat Earth Society) in casual conversation, but when I consider how much I've read about and discussed the DK effect, I can't say that the implications as stated in that the quoted piece actually has a significant basis in reality. In other words, DK may be the "darling" of said group, but the self-same group is not very significant in any way in my experience. YMMV.
- sova 6y agoIsn't the Dunning-Kruger effect about personal mastery, and not comparative social science?
- devilduck 6y agoTo me it ends up feeling like people who say things like "people are stupid" and use the Dunning-Kruger effect to hand wave away harder socio-psychological problems are projecting their own insecurities of their lack of knowledge and can be safely ignored
- jonnycomputer 6y agoCould the article actually getting around to explaining the simulation using random data? Frustrating to read the things, and then just be shown a graph purportedly from random data and not explain how it was generated. Is it an artifact of using quartiles?
- ajarmst 6y agoWait a sec’. The author admits the effect has been confirmed by others, then claims it’s probably not real because of a pair of studies (both from the same team) that even the authors admit support at least part of the Dunning-Kruger hypothesis (“It’s just that experts do that over a narrower range”). His confirmation is a couple of ad hoc unpublishable investigations? It’s almost like he might be overestimating his ability to evaluate the work. In particular, he seems to be more interested in refuting a popular perception of what the effect is rather than the more subtle claims that Dunning and Kruger actually made—another source of the type of errors predicted for those without expertise in a given field.
- ajarmst 6y agoOf course, an attack on Dunning Kruger demonstrating questionable competence to evaluate their work is delightfully meta.
- Lxr 6y agoThe article doesn't explain what is actually going on to produce the second chart, but here is a guess. Assume people have a true ability x, an estimated ability y which is x + some unbiased noise, and a test score z which is also x + some unbiased noise. If you take a sample and collect the lowest quartile of test scores z, then the average of the estimated abilities y in that group is higher than z because the y are produced centered at x rather than centered at z. By collecting the lowest quartile of test scores, you didn't just get the dumbest people, you also got the people who happened to perform badly on the test that day. These people may be estimating their ability accurately, in which case the estimate would be higher than their randomly bad performance that day. This seems like too obvious a mistake to not have been noticed for this long though.
- scotty79 6y agoSo it's just artifact of testing. Lowest test scores come from low ability plus bad luck and as such are lower than just low ability and highest test scores come from high ability and great luck and as such are higher than just high ability.
- AstralStorm 6y agoEasily verified by testing more. (Via law of big numbers.) Unless the test is biased which means we're back to square one - discerning bias of the test from bias of respondent. To do that you'd have to produce a specifically biased test in a different way, see how the noises add.
- jonnycomputer 6y agoUsing quartiles like this is probably ill advised. Just show the scatter.
- SomewhatLikely 6y agoHowever, from my reading of the article they were asked to estimate their performance after they had already taken the test, not their inherent ability.
- univalent 6y agoNot enough evidence in the paper. And does anyone feel like this effect is not real in this year of all years? People were screaming 'fake news', 'fake virus', 'fake election', blah blah with great conviction.
- tyu2 6y agoSearch HN for the effect, you'll see criticism of people citing it from years back, so it's not this year's thing at all.
- lrossi 6y ago> instead showed that both experts and novices underestimate and overestimate their skills with the same frequency. “It’s just that experts do that over a narrower range,” he wrote to me. This makes sense, in the narrow definition they use for the effect, that is students estimating their exam score. It doesn’t necessarily disprove other interpretations that people are casually using, such as imposter syndrome, which are related but pedantically not the same effect.
- gnzoidberg 6y agoIt's most definitely real
- Shacklz 6y agoNot sure if the Dunning-Kruger effect can be applied to everything (such as: humour, grammar, etc., as listed in the article), but I think pretty much everyone who has played any kind of video game (Dota, LoL, Star Craft, CS:GO, etc.) in a competitive "try-hard" environment for long enough will assure you that the Dunning-Krueger effect is indeed real. "It's not my fault for losing this; I am clearly better than the team/oponents if it weren't for bad teammates/cheese tactics/meta abuse/lags/etc" is a mindset a lot of players have even if all evidence points to the contrary (~50% winrate in average in matches with players of similar ranking etc.). In fact, I struggled quite some time with this myself - acknowledging and understanding one's own shittiness is one of the very first steps of actual improvement, and that's usually easier said than done, at least for me personally.
- NiceWayToDoIT 6y agoSame goes for IT projects, junior developers usually have a significant confidence when facing new problem (task estimation). Their confidence sometimes comes from limited ability to see all the path that may go wrong, on the other hand senior developers although have considerable experience, sometime overthink solution because of the past experiences, and often they leave extra space for so called "known unknowns" and "unknown unknowns".
- 1123581321 6y agoWhile I've certainly seen players say those things, and agree that increasing humility is a good corrective to that character flaw, that kind of public bragging isn't indicative of their self-evaluation, and I haven't seen that expert players necessarily stop that behavior or demonstrate that humility was the key to their skill mastery.
- apple4ever 6y agoBut you need to differentiate what people outwardly project is their ability and what they actual think is their ability. And then there is also the reaction in the moment vs the reaction after sometime of thinking about it.
- arithma 6y agoWe all at least are dumber than we'd like to be. Nobody would refuse a pill that would make them smarter. Sorry for off-topic, but this has been gnawing at me lately, feels like something is a miss about my cognition, only very subtle, nothing to go for any kind of practitioner. Is there any route actually for giving the good old fat bucket a rattle to make it work back at original capacity? Have you ever tried anything? I know this is off-topic, if it annoys anyone, I could take it somewhere else, but thought people interested in this article might have experimented or self-reflected.
- djohnston 6y agoThere wasn't an actual explanation as to WHY it wouldn't be real. They show the original plot, then another plot generated with random data that looks like a dampened version of the original effect. Then there are like 10 paragraphs on either side of these two pictures, which don't do a lot to support these critical plots. I am left unsatisfied! > "The above Dunning-Kruger graph was created by Patrick McKnight using computer-generated results for both self-assessment and performance. The numbers were random. There was no bias in the coding that would lead these fictitious students to guess they had done really well when their actual score was very low. And yet we can see that the two lines look eerily similar to those of Dunning and Kruger’s seminal experiment. A similar simulation was done by Dr. Phillip Ackerman and colleagues three years after the original Dunning-Kruger paper, and the results were similar." OK, so you removed the brain, and made some models that produce somewhat similar results. What if your models just modeled the brain? How does this refute the original?
- virtualritz 6y ago>[...] both experts and novices underestimate and overestimate their skills with the same frequency. Looking at these graphs there is maybe some curious conclusion to be made about the symmetry with imposter syndrome. If we believe to have 'average' (2nd/3d quartile) knowledge about a subject, there are three possibilities: 1. We are correct in this assumption. 2. We are victims of Dunning-Kruger and know less than we believe we do. 3. We are victims of imposter syndrome and know more than we believe we do.
- TimTheTinker 6y agoI think there are excellent intuitive arguments in favor of the Dunning-Kruger effect. One is that university students often talk about how suddenly ignorant they feel once they start taking classes with depth of content — and the Ph.D. students who say the feeling only increases the deeper you go.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- ml_thoughts 6y agoThis code seems to somewhat re-create effect, except noise is symmetric at both sides (also true of the xkcd-style plot in this article, actually). Plot: https://imgur.com/a/dGZyylf https://imgur.com/a/dGZyylf It seems a crucial piece of context is that there is some correlation between perceived and actual performance. There also is a global optimistic bias, apparently across experiments people perceive their rank is in the 66th percentile, so this universal effect leaves more room for error on the under-performing people while the highly-performing people are going to end up closer to that 66th percentile tautologically, and bounded above by smaller amount of space. import random import numpy as np np.random.seed(seed=12345) from scipy.linalg import eigh, cholesky from scipy.stats import norm from matplotlib import pyplot as plt #Draw correlated random variables #Ref here: https://scipy-cookbook.readthedocs.io/items/CorrelatedRandomSamples.html num_samples = 125 * 4 # x = norm.rvs(size=(2, num_samples)) # uncorrelated random normal variables expected_correlation = np.array([[1.0, 0.19], [0.19, 1.0]]) c = cholesky(expected_correlation, lower=True) #there's a slight correlation of R = 0.19 between actual and perceived scores (according to Ackerman, 2002) trans_x = np.dot(c, x) #perceived / actual readings with R=0.19 perceived = trans_x[0, :] actual = trans_x[1, :] #Sort both variables by actual scores. sort_by_actual = sorted(range(num_samples), key = lambda idx: actual[idx]) perceived_by_actual = [perceived[i] for i in sort_by_actual] actual_by_actual = [actual[i] for i in sort_by_actual] quartile_indices = [i * (num_samples // 4) for i in range(5)] #note: depends on divisiblity by four x_coords = [xx // 2 for xx in quartile_indices[1:]] #mid-points just for plotting perceived_means = [np.mean(perceived_by_actual[start:end]) for (start, end) in zip(quartile_indices[:-1], quartile_indices[1:])] actual_means = [np.mean(actual_by_actual[start:end]) for (start, end) in zip(quartile_indices[:-1], quartile_indices[1:])] #Plot fig = plt.figure() ax1 = fig.add_subplot(111) plt.title("Dunning-Kruger") ax1.scatter(x_coords, perceived_means, marker="s", label="perceived") ax1.scatter(x_coords, actual_means, marker="o", label="true") ax1.legend() plt.show()
- tpoacher 6y ago1. Dunning-Kruger effect is typically defined as X. 2. Here, we define it as Y. 3. We show that Y is false. 4. Therefore X is false. Uh, no.