4 ms·

The problem is the incentives. To do well, you must publish. To publish, you must have a good story, and ‘we tried this and it didn’t work’ is not one. So afte

The problem is the incentives. To do well, you must publish. To publish, you must have a good story, and ‘we tried this and it didn’t work’ is not one. So after a certain time spent, you are left with a choice of ‘massaging’ the data to get some results, or not and getting left behind those that do or were luckier in their research.

hef19898 5y ago
"We tried this and it didn't work, and here's why we think it didn't" should be among the bests stories to publish. Looking back I learned more from stuff that didn't work, or rather figuring out why it didn't, than from success.
- Frost1x 5y ago
  Tell that to the people who pay for research and the metrics they use to continue feeding those who perform research. Ultimately, this is the problem.
- j7ake 5y ago
  Even if something did not work, you still need a story for it to be readable. For example, I imagine that archeological work is extremely high impact if excavation efforts led to discovery of ancient city. Archeology paper would probably be less interesting if the paper said “we dug this area, found nothing”. If one were to judge those two papers, obviously the discovery paper is higher impact than the negative result.
  - dtech 5y ago
    "We chose this area because we believed it should be archeologically interesting based on XYZ. However, we searched through ABC methods and found nothing there" would be valuable for the future. Maybe XYZ isn't as good as we though, maybe ABC couldn't find it. Maybe now some other sod in the future won't try that location. Not as valuable as a discovery, but very far off zero value. Yet the reward in academia would be near-zero.
- dagw 5y ago
  or rather figuring out why it didn't That can end up being just as time consuming as doing the research to begin with. Often there is no time and no money to go back and do that. If your 'budget' is 6 month you're going to spend 6 month trying to get your experiment to work. You're not going to 'give up' after 4 month and spend 2 month putting together a "why we failed" paper.
  - bumby 5y ago
    However, the advantage is, if it is published, it can decrease the likelihood of multiple other attempts to try the same “unique” (and wrong) approach.

hef19898 5y ago

"We tried this and it didn't work, and here's why we think it didn't" should be among the bests stories to publish. Looking back I learned more from stuff that didn't work, or rather figuring out why it didn't, than from success.

Frost1x 5y ago
Tell that to the people who pay for research and the metrics they use to continue feeding those who perform research. Ultimately, this is the problem.
j7ake 5y ago
Even if something did not work, you still need a story for it to be readable. For example, I imagine that archeological work is extremely high impact if excavation efforts led to discovery of ancient city. Archeology paper would probably be less interesting if the paper said “we dug this area, found nothing”. If one were to judge those two papers, obviously the discovery paper is higher impact than the negative result.
- dtech 5y ago
  "We chose this area because we believed it should be archeologically interesting based on XYZ. However, we searched through ABC methods and found nothing there" would be valuable for the future. Maybe XYZ isn't as good as we though, maybe ABC couldn't find it. Maybe now some other sod in the future won't try that location. Not as valuable as a discovery, but very far off zero value. Yet the reward in academia would be near-zero.
dagw 5y ago
or rather figuring out why it didn't That can end up being just as time consuming as doing the research to begin with. Often there is no time and no money to go back and do that. If your 'budget' is 6 month you're going to spend 6 month trying to get your experiment to work. You're not going to 'give up' after 4 month and spend 2 month putting together a "why we failed" paper.
- bumby 5y ago
  However, the advantage is, if it is published, it can decrease the likelihood of multiple other attempts to try the same “unique” (and wrong) approach.