3 ms·
I think this is largely a case of 'moloch' as described by Scott Alexander: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ http://slatestarcodex.co
by Fede_V 10y ago
I think this is largely a case of 'moloch' as described by Scott Alexander: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ http://slatestarcodex.com/2014/07/30/meditations-on-moloch/
In that long (but excellent essay) Scott points out that self organizing systems like human society can sometimes arrange themselves in horrible local minimas which can be very difficult to escape.
I think the way modern science is organized is decidedly suboptimal. Note that I'm talking purely sociologically: science, as a method, is still by far the best thing we've developed to understand the natural world. However, the incentives around publishing / grants / hiring work are broken in a myriad of ways, from small to massive.
For example:
- High prestige journals/conferences are basically a crapshoot: http://blog.mrtz.org/2014/12/15/the-nips-experiment.html http://blog.mrtz.org/2014/12/15/the-nips-experiment.html - consider that getting a paper accepted in NIPS/published in Nature might completely change the way your career shapes up.
- Grant funding is incredibly competitive, and the way they are awarded is also a crapshot: https://psmag.com/why-the-national-institutes-of-health-should-replace-peer-review-with-a-lottery-41d09bcce5d8 https://psmag.com/why-the-national-institutes-of-health-shou...
- The pressure to publish/obtain grants drives people to either make up data to obtain fancy publications, or even to kill themselves: http://www.dcscience.net/2014/12/01/publish-and-perish-at-imperial-college-london-the-death-of-stefan-grimm/ http://www.dcscience.net/2014/12/01/publish-and-perish-at-im...
- We reward researchers for brilliant clean discoveries, not brilliant methodologies. However, the outcome of a serious research project is the one thing that a scientist cannot control: science investigates the unknown, if someone explores a plausible hypothesis in a clever way, and the hypothesis turns out to be wrong, the scientist didn't do anything wrong.
- PhDs are often used as cheap labour. A very famous PI once bragged that before they got their new fancy robot, they just had 10 chinese postdocs handle all the plating.
- Private companies make an insane amount of money from publishing scientific research that's carried out with public money, and that's reviewed by scientists (largely) paid with public money who donate their money for free.
I could go on for ages - and so could almost every researcher/ex-researcher on HN. There's a few no brainer changes - but the problem is that almost everyone in power who could affect change stands to benefit from the current system.
- Strilanc 10y agoA more relevant post by Scott is The Control Group is Out of Control [1]. The starting off point is that parapsychology has high-quality studies with positive results, but we're pretty sure parapsychology is wrong, and this bodes poorly for "high-quality" studies in other fields. But then it just keeps going and going... by the end I always feel like lowering the acceptable p-value ceiling by a factor of 1000, and also like that would be a joke laughed off by the nature of the problem. 1: http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...