11 ms·

> The reality is that the study was looking at the difference best and worst performers, not best vs average, which is an important clarification. That is an

> The reality is that the study was looking at the difference best and worst performers, not best vs average, which is an important clarification. That is an important, often misunderstood, clarification, but I think it’s also subtly incorrect. It’s 10x higher than some minimally competent level, not 10x higher than the worst in the field. I think the original studies (certainly later follow-up studies did this) excluded results from participants who did not complete the assignment at all. That makes some sense from a data analysis convenience standpoint, but truncating the left tail and then saying some are 10x better than the absolute worst (that you truncated in the previous step) isn’t fully representative of what we see in the field.

xnorswap 2y ago
In any case, people on here tend to be extremely sceptical of social science results, and demand much better standards and reproducibility. Except that particular 1968 paper, which is held up and treated as gospel, because it helps to confirm rather than challenges their personal experience. If we are going to demand better, we should do so consistently. Reproducing that result with more rigour would be a useful start.
- sokoloff 2y ago
  I group the skepticism that I see around this topic into two buckets. One is "it's not 10x, but is maybe 4 or 5x"; the other is "if differences exist at all, it's definitely less than 2x". I get/have sympathy and agreement for the first type on a technically-precise level, but it also doesn't really change any of my behavior as a technologist or people/org leader. The second type of skepticism I just don't understand at all from people who have worked more than a couple years in industry.