4 ms·

The time horizon that publish or perish forces is not conducive to high variance research. If you do a year of research and get a null result you are screwed, s

The time horizon that publish or perish forces is not conducive to high variance research. If you do a year of research and get a null result you are screwed, so what do people do? Not pursue research with a significant chance of that happening. But high variance research is exactly what leads to major breakthroughs.

kkylin 10y ago
I agree with what you said, but high variance research means high variance outcomes... not just scientific but also professional, and not just oneself but also one's grad students and so on. It is a chance we should all take from time to time, but not without calculating the odds and potential consequences, good or bad...
nonbel 10y ago
>"If you do a year of research and get a null result you are screwed" The best I heard on this site was that "alpha is the expected value of p". After a few years of calibration, each field collects just enough data (or adjusts the cutoff) to pump out "statistically significant" results at the appropriate rate. It makes it just hard enough to seem like an accomplishment, but most grad students will get something to publish if they just keep at it. Of course, that is a totally arbitrary metric that has little to do with the quality of the research.
- Houshalter 10y ago
  Somewhat relevant: http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ http://slatestarcodex.com/2014/04/28/the-control-group-is-ou... We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist. And they still manage to produce tons of significant results and findings, despite using much higher standards than regular social sciences.
  - taxicabjesus 10y ago
    > We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist. That this "phenomenon" does not and cannot exist is the foundational assumption of modern science. Back in the days when the Laws of Thermodynamics were being figured out to describe the properties of steam engines, a few chemists realized that everything in the universe could be described in terms of indivisible 'atoms'. Atoms aren't indivisible anymore, but the assumption lives on. Life is much more interesting if one assumes that our 3d-reality is but a projection of a higher-ordered reality.

kkylin 10y ago

I agree with what you said, but high variance research means high variance outcomes... not just scientific but also professional, and not just oneself but also one's grad students and so on. It is a chance we should all take from time to time, but not without calculating the odds and potential consequences, good or bad...

nonbel 10y ago

>"If you do a year of research and get a null result you are screwed" The best I heard on this site was that "alpha is the expected value of p". After a few years of calibration, each field collects just enough data (or adjusts the cutoff) to pump out "statistically significant" results at the appropriate rate. It makes it just hard enough to seem like an accomplishment, but most grad students will get something to publish if they just keep at it. Of course, that is a totally arbitrary metric that has little to do with the quality of the research.

Houshalter 10y ago
Somewhat relevant: http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ http://slatestarcodex.com/2014/04/28/the-control-group-is-ou... We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist. And they still manage to produce tons of significant results and findings, despite using much higher standards than regular social sciences.
- taxicabjesus 10y ago
  > We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist. That this "phenomenon" does not and cannot exist is the foundational assumption of modern science. Back in the days when the Laws of Thermodynamics were being figured out to describe the properties of steam engines, a few chemists realized that everything in the universe could be described in terms of indivisible 'atoms'. Atoms aren't indivisible anymore, but the assumption lives on. Life is much more interesting if one assumes that our 3d-reality is but a projection of a higher-ordered reality.