4 ms·

The pursuit of novelty in research drives me crazy. You don't get published as easily with replications and tiny iterative developments. I wish they'd incentivi

by acbart 11y ago

The pursuit of novelty in research drives me crazy. You don't get published as easily with replications and tiny iterative developments. I wish they'd incentivize those kinds of research...

SapphireSun 11y ago
I think you have the opposite opinion of most of the researchers I know. They go nuts that you can't do anything really out of the box until you have tenure. We agree on replication though. No one cares until there's a crisis, then the Karl Popper quotes never stop.
- danieltillett 11y ago
  You are allowed to be as out of the box as you like, but you have to generate results which if very hard to do if you take risks. Even once you have tenure you are still forced to still play it safe since if you don’t generate the results you don’t get grants and if you don’t bring in the money your university will find ways to get rid of you. Welcome to your new lab - yes we know it was the old broom closet, but you must understand that since you don’t have any grants right now we need to give your lab to research x who has just brought in a $3 million grant.
danieltillett 11y ago
We certainly undervalue replication (or refutation) in the pursuit of novelty. What is more of a problem is that scientists are being pushed by the system to get their results out as fast as possible so they are not doing the hard work of carefully checking their own work before publication. The incentive is to publish if an experiment works once or twice and hand-wave away the other dozen times it didn’t. Combine that with the enormous pressure PostDocs and PhD students are under to generate results it is amazing anything of value comes out of the lab these days.
- Sven7 11y ago
  The number of people capable of producing good science, is much less than the number of people in science. The pressures emerge because of it.
  - danieltillett 11y ago
    Even the people capable of producing good science are pushed into bad situations. The basic problem is we have more good scientists than the funding to support them - when the success rate of grants is less than 15% then the pressure gets extreme and also sorts of bad behaviour results. As a bit of an aside grant reviews were apparently brought in initially to help scientists avoid working on bad areas. The idea was that everyone would get funded who put up a reasonable grant and they just wanted to help screen out the bad ideas so people didn’t waste their time. We have since twisted this into the current system where we try to determine which of the top few percent of grants are the best. Of course the whole ranking process is so noisy that it is little better than chance for anyone in the top 25%. A much better system would be to rank the grants as worth funding or not and then put all those worth funding into a lottery to choose who gets funded. At least this way even the losers would know they were on the right track and they were just unlucky. It would save an enormous amount of grantmanship and other activities that are the antithesis of everything science should be about.

by acbart 11y ago

The pursuit of novelty in research drives me crazy. You don't get published as easily with replications and tiny iterative developments. I wish they'd incentivize those kinds of research...

SapphireSun 11y ago

I think you have the opposite opinion of most of the researchers I know. They go nuts that you can't do anything really out of the box until you have tenure. We agree on replication though. No one cares until there's a crisis, then the Karl Popper quotes never stop.

danieltillett 11y ago
You are allowed to be as out of the box as you like, but you have to generate results which if very hard to do if you take risks. Even once you have tenure you are still forced to still play it safe since if you don’t generate the results you don’t get grants and if you don’t bring in the money your university will find ways to get rid of you. Welcome to your new lab - yes we know it was the old broom closet, but you must understand that since you don’t have any grants right now we need to give your lab to research x who has just brought in a $3 million grant.

danieltillett 11y ago

We certainly undervalue replication (or refutation) in the pursuit of novelty. What is more of a problem is that scientists are being pushed by the system to get their results out as fast as possible so they are not doing the hard work of carefully checking their own work before publication. The incentive is to publish if an experiment works once or twice and hand-wave away the other dozen times it didn’t. Combine that with the enormous pressure PostDocs and PhD students are under to generate results it is amazing anything of value comes out of the lab these days.

Sven7 11y ago
The number of people capable of producing good science, is much less than the number of people in science. The pressures emerge because of it.
- danieltillett 11y ago
  Even the people capable of producing good science are pushed into bad situations. The basic problem is we have more good scientists than the funding to support them - when the success rate of grants is less than 15% then the pressure gets extreme and also sorts of bad behaviour results. As a bit of an aside grant reviews were apparently brought in initially to help scientists avoid working on bad areas. The idea was that everyone would get funded who put up a reasonable grant and they just wanted to help screen out the bad ideas so people didn’t waste their time. We have since twisted this into the current system where we try to determine which of the top few percent of grants are the best. Of course the whole ranking process is so noisy that it is little better than chance for anyone in the top 25%. A much better system would be to rank the grants as worth funding or not and then put all those worth funding into a lottery to choose who gets funded. At least this way even the losers would know they were on the right track and they were just unlucky. It would save an enormous amount of grantmanship and other activities that are the antithesis of everything science should be about.