3 ms·
All it takes is 14 grad students studying the same thing targeting a 95% confidence interval for, on average, one to stumble upon a 5% case. Factor in publicati
by dehrmann 1y ago
All it takes is 14 grad students studying the same thing targeting a 95% confidence interval for, on average, one to stumble upon a 5% case. Factor in publication bias and you get a bunch of junk data.
I think I heard this idea from Freakonomics, but a fix is to propose research to a journal before conducting it and being committed to publication regardless of outcome.
- beng-nl 1y agoA great idea. Also known as a pre registered study. https://en.m.wikipedia.org/wiki/Preregistration_(science) https://en.m.wikipedia.org/wiki/Preregistration_(science)
- poincaredisk 1y agoNot familiar with this idea, but this idea is commonly applied for grant applications: only apply for a grant when you finished the thing you promise to work on. Then use the grant money to prototype the next five ideas (of which maybe one works), because science is about exploration.
- mikeyouse 1y agoMost pharma / medicine studies are pre-registered now. Sometimes the endpoints change based on what the scientists are seeing, but if they're worth their salt, they still report the original scoped findings as well.
- constantcrying 1y ago>but a fix is to propose research to a journal before conducting it and being committed to publication regardless of outcome. Does not fix the underlying issue. Having a "this does not work" paper on your resume will do little for your career. So the incentives to make data fit a positive hypothesis are still there.
- sightbroke 1y agoThat is categorically not true. Showing why something does not work (or is not advantageous over other methods) demonstrates you know how to properly conduct research which is good for ones resume.
- constantcrying 1y agoThe paper is irrelevant and will never get cited. There is essentially zero benefit to your career as it is nothing more than a single bullet point on your resume. Discovering something that works is significant, discovering something that does not work is irrelevant. Can you name a single scientist, e.g. from your field, who is known for showing that something does not work?
- sightbroke 1y ago[flagged]
- constantcrying 1y agoI think I am better informed than most of the population. How many research papers do you read per month?
- sightbroke 1y agoI read papers quite regularly as part of a function of my job.