3 ms·

Alternate title: “10k-hour rule confirmed though not quite as well as originally hoped” Much better link to the full text of the actual replication study, wit

by cwillu 3y ago

Alternate title: “10k-hour rule confirmed though not quite as well as originally hoped” Much better link to the full text of the actual replication study, without the vox-splaining: https://royalsocietypublishing.org/doi/10.1098/rsos.190327 https://royalsocietypublishing.org/doi/10.1098/rsos.190327 From the abstract: “Overall, the size of the effect was substantial, but considerably smaller than the original study's effect size.” Regarding small sample sizes: “ The current sample was obtained from over 4 years of recruiting efforts (summer of 2014–autumn of 2018). While our sample size is 33% larger than Ericsson et al.'s, it is still small. Conducting expertise research, by definition, means studying a small subset of the population, thus there are relatively few participants from which to sample, making large-sample replications nearly impossible. However, if replications of expertise studies never enter the scientific record because of small samples, then we probably will never provide additional evidence to support or refute the original studies. This leaves only the original study (with an even smaller sample size) as the only indicator in the scientific record.” “To be clear, evidence from small samples—in original studies or replications—should be interpreted with caution. However, when the original study (with a small sample) has already entered the scientific record, replications with similar sample sizes (since that is all that is feasible) should also be allowed to enter the scientific record rather than be suppressed from publication. This is probably the best way to accumulate knowledge in this area. ” Oooo! And they link to actual data they collected in the study! https://osf.io/4595q/ https://osf.io/4595q/ Fun little chart from midway through the paper: https://royalsocietypublishing.org/cms/asset/bc80eddb-371e-47cc-b466-de737d047188/rsos190327f01.gif https://royalsocietypublishing.org/cms/asset/bc80eddb-371e-4...

readthenotes1 3y ago
"replications with similar sample sizes (since that is all that is feasible) should also be allowed to enter the scientific record rather than be suppressed from publication." Fake it 'til you make it. That should be the slogan of the modern scientific process...
- eesmith 3y ago
  A preregistered double-blind study, as this was, is about as far from "fake it 'til you make it" as you can be. Actually, this is one step even better - version 1 of the manuscript, expect for results and discussion, was also preregistered. The old - and wrong - scientific process was "only publish what is novel", leading to publication bias.
oersted 3y ago
> Practice still mattered: It accounted for 26 percent of the difference between good violinists and the less accomplished students. But the original study claimed that practice accounted for 48 percent of the difference. This is very different from "10k-hour rule confirmed". The study concludes that practice is positive, but it is a small part of being the best, practicing a lot won't make much of a difference to achieving elite status, although it is probably a requirement for it. The study also points out that the top players actually had practiced less than the average of good players. And this is for violin playing (plus another meta-analysis on sports). While the 10k-hour rule is most often associated to Bill Gates, and technical, academic or business success (at least in HN circles). Very different domains where physical dexterity matters little and learning processes have nothing to do with internalizing complex movement. The numbers discussed in the study are roughly around the 10k-hour ballpark (practice before the age of 18). But again, there's nothing indicating that this generalizes to anything other than violin playing. I think it is fairly obvious that different disciplines take different time to master, although I haven't looked into the studies about it.
- mc32 3y ago
  Aren’t many of these populations self selecting anyway? It would be something else if the took a random sample of people off the street, got them to use ~10,000 hours to master something and then got positive results.

by cwillu 3y ago

readthenotes1 3y ago

"replications with similar sample sizes (since that is all that is feasible) should also be allowed to enter the scientific record rather than be suppressed from publication." Fake it 'til you make it. That should be the slogan of the modern scientific process...

eesmith 3y ago
A preregistered double-blind study, as this was, is about as far from "fake it 'til you make it" as you can be. Actually, this is one step even better - version 1 of the manuscript, expect for results and discussion, was also preregistered. The old - and wrong - scientific process was "only publish what is novel", leading to publication bias.

oersted 3y ago

> Practice still mattered: It accounted for 26 percent of the difference between good violinists and the less accomplished students. But the original study claimed that practice accounted for 48 percent of the difference. This is very different from "10k-hour rule confirmed". The study concludes that practice is positive, but it is a small part of being the best, practicing a lot won't make much of a difference to achieving elite status, although it is probably a requirement for it. The study also points out that the top players actually had practiced less than the average of good players. And this is for violin playing (plus another meta-analysis on sports). While the 10k-hour rule is most often associated to Bill Gates, and technical, academic or business success (at least in HN circles). Very different domains where physical dexterity matters little and learning processes have nothing to do with internalizing complex movement. The numbers discussed in the study are roughly around the 10k-hour ballpark (practice before the age of 18). But again, there's nothing indicating that this generalizes to anything other than violin playing. I think it is fairly obvious that different disciplines take different time to master, although I haven't looked into the studies about it.

mc32 3y ago
Aren’t many of these populations self selecting anyway? It would be something else if the took a random sample of people off the street, got them to use ~10,000 hours to master something and then got positive results.