4 ms·
Preregistration, requirement to publish negative or null results, and full data is, arguably, the three legs of modern science. If we collectively don't enforce
by DiabloD3 5y ago
Preregistration, requirement to publish negative or null results, and full data is, arguably, the three legs of modern science. If we collectively don't enforce this, nobody is doing science, they're just fucking around and writing it down.
- romwell 5y agoAlso replication studies for negative or null results in addition to positive ones (we don't have either).
- candiodari 5y agoYou do realize there is a million negative results for every one positive result? This is equally easy to game, maybe easier.
- naasking 5y agoYes, and knowing what's been tried and what has failed is important.
- TOMDM 5y agoI think what's being pointed out is that "researchers" could pump out hundreds of easy to test negatives every day if a negative result was just as incentivised. I do agree though, negatives are just as important when the intent is to prove/disprove a meaningful hypothosis.
- kuhewa 5y agoA negative result won't make a career. I don't think there's much danger when requiring negative results going onto a repository of over incentivising negative results. You can't mandate Nature or Cell publishes negative results.
- hprotagonist 5y agowe tried using 0.1 mL, it didn't work we tried using 0.11 mL, it didn't work we tried using 0.12 mL, it didn't work we tried using 0.13 mL, it didn't work
- NavinF 5y agoIf someone really tested those hypotheses, let them publish. I doubt they'll get funding so it'll be on their own dime. In practice people do run experiments like that, but they only publish the 1/4 trials that is successful.
- naasking 5y agoLook to physics for how negative results should be published. There typically has to be reason to suspect some dosage range should work, in which case that sequence of studies you describe would be perfectly valid if it's within that range.
- harry8 5y agowe tried using 0.10 mL, it didn't work we tried using 0.11 mL, it didn't work we tried using 0.13 mL, it didn't work we tried using 0.15 mL, it didn't work we tried using 0.17 mL, it didn't work we tried using 0.16 mL, it didn't work we tried using 0.18 mL, it didn't work we tried using 0.20 mL, it didn't work we tried using 0.14 mL, it didn't work we tried using 0.12 mL, it worked so we published Do you want to know the ones that "didn't work" existed? Or are you happy with just the one that "worked" being written up in isolation?
- hprotagonist 5y agoi don’t want to know about each test that didn’t work as a separate publication, that’s for sure!
- carlmr 5y agoThat's true, we would need a way to collect this data so it's easily seen as part of a whole. E.g. if you search for eggs and cholesterol you should find all studies with their summarized results on whether eggs are ok or not for your cholesterol, grouped by researcher, so if somebody does 200 studies to find the one positive it's instantly visible.
- captain_price7 5y agoWhy would someone want to game a negative result? Nobody ever becomes famous for saying my approach doesn't work. (As long as science is open, to make sure there is actually good work done by researchers before reaching this neg result.)
- candiodari 5y agoTo have their name on a publication, which is a currency in the academic world.
- divbzero 5y agoI like rules like these. One context where preregistration, null results, and full data are all required are clinical trials overseen by the FDA. It’s no surprise that those studies carry a lot of weight.