5 ms·
The article reminded of me of Robert Axelrod's excellent "The Evolution of Cooperation." Axelrod uses similar computer modeling to create a tournament to help t
by ChicagoBoy11 10y ago
The article reminded of me of Robert Axelrod's excellent "The Evolution of Cooperation." Axelrod uses similar computer modeling to create a tournament to help tease out how cooperative strategies emerge from primarily self-interested behavior. It is one of the best things I have ever read in my life, and I'd recommend that book to anyone.
The real issue with scientific publishing is that there is simply no penalty for publishing shoddy research. I know several academics who made quite a big name for themselves on research that was later partially or fully retracted. No one cared about that; there was no real reputational damage done. To tackle poor science, such "poor" scientific inquiry should be "punished" in some way. Similarly, it is terrible for the advancement of science that only novel or significant results get published -- there should be a way for researchers to benefit from publishing well-designed research which simply did not yield interesting results.
How to do that? I think some of Axelrod's tournament provides an answer. Like in his examples, the individual incentives align to yield a pretty poor outcome to the members of his population (he runs an iterated prisoner's dilemma game). However, correctly setting up the iteration parameter's, slowly a cooperative strategy becomes the evolutionarily stable strategy.
I can see how this could also be the case for academics. There is no law from up above that dictates that "number of papers published" is the ultimate metric of success. There is a culture, and processes, and institutions which have led that to be a leading indicator of academic success. If there were real motivation and impetus to change this, there is no reason to imagine that other metrics (and processes) could emerge that would much more highly value scientific integrity and thoroughness.
- j2kun 10y ago> I know several academics who made quite a big name for themselves on research that was later partially or fully retracted. No one cared about that; there was no real reputational damage done. cf. the "power pose"
- YeGoblynQueenne 10y agoIt's hard to find information on what the problem was with that one, so here's a link to an article from Slate: http://www.slate.com/articles/health_and_science/science/2016/01/amy_cuddy_s_power_pose_research_is_the_latest_example_of_scientific_overreach.html http://www.slate.com/articles/health_and_science/science/201...
- erichocean 10y agoIronically, "failure to replicate" can also be spurious.
- j2kun 10y agoIIRC there were on the order of ten attempted replications, all of which failed.
- infectoid 10y agoI have heard a suggestion that PHD eligibility should require a replication. This would not only help validate (or invalidate) studies but would also over time reinforce the importance of replication culturally.
- red_dazzler 10y agoYea, because it is really all the overworked and underpaid graduate students faults... I have never met a single graduate student who rushed to publish shoddy work on his/her own. However, I have watched time and again while young impressionable graduate students are slowly but surely guided to publish shoddy work by overworked and desperate professors who feel they have no choice but to keep getting "academic currency" (e.g. publications).
- afarrell 10y agoI think GP is saying that you should have to replicate another's work, not that your dissertation must be replicated. The latter would tack on I think we should make it more precise: in order to reach <stage>, you should have to replicate another's work that has not been replicated more than three times as of 6 months ago and produce either a record of a successful replication or a bug report detailing a failed replication. As a best practice, a lab where few of the incoming grad students can produce a definitive (either successful or failed) replication should checked-on by the university to see if they've just got a terrible working environment.
- sevensor 10y agoThis would be great training, too. Not only would grad students learn how to conduct experiments in their field, they would also learn the value of a good Methods section in a paper.
- jclos 10y agoI don't know if making replication a "grad student chore" that you need to get through is a good way to remove the stigma behind "wasting" time replicating other's people scientific results instead of building your own. Perhaps we should just admit that replicating scientific results is just as much science as creating original work is, and grant degrees to students who manage to replicate a lot of work and publish it in a rigorous analysis.
- anewhnaccount 10y agoRichard Dawkins presented this research in a quite enjoyable pop-sci documentary in the 80s: http://www.dailymotion.com/video/x1jprgh_bbc-horizon-nice-guys-finish-first-richard-dawkins-1987_tech http://www.dailymotion.com/video/x1jprgh_bbc-horizon-nice-gu...