3 ms·
Fixing Science - Systems and Politics
- diminish 15y agoThe notion of a central registry and government regulation for science is horrific and backwards no matter what the initial intention is. Why not a distributed, peer-to-peer approach for scientific publishing where all articles are anonymously accessed and linked to each other?
- pessimizer 15y agoBecause that, in no way, will prevent researchers from trashing studies until they get the results they want, or fishing for any significant correlation after the fact as if that's what they were looking for all along. Some sort of validation mark that a study announced itself before being started would at least show me which ones I should ignore - because IMO this type of thing is ruining medicine, esp. pharma. At this point, there's very little evidence that some major drugs, classes of drugs, and treatments work at all, and that's both scary and a systemic financial drain.
- hobin 15y agoI think it's a rather misleading post. He cites a paper from PLoS Medicine, and I suspect that those without a good background in research would rather quickly jump to the conclusion that "it's in a science journal, so it must be (very likely to be) true." However, it should be noted that this is an essay, which PLoS Medicine defines as opinion pieces on a topic of broad interest to a general medical audience. More importantly, lets look at this from a pragmatic point of view. Are there things that can be done better in science? Absolutely. Does science somehow 'not work'? Not at all. Does it need fixing? No, why should it? Anyone who suggests that science is fundamentally broken must've lived under a rock for the past decades, or at least hasn't been keeping an eye on what science has achieved. I also disagree with the blog author's 'solution' to the problem. He (or is it a she? I don't know) argues in favor of a centralized approach to science, whereas one of the reasons science is so successful is because it's so decentralized!
- yummyfajitas 15y agoDoes it need fixing? No, why should it? If science could be done better, yes, we should fix it. The author identifies a very real problem - publication bias and data mining, and proposes a solution, namely pre-registration of experiments. Only one of his mechanisms for enacting the solution (#1, "clinicaltrials.gov option") is centralized.
- disgruntledphd2 15y agoUm, on your point about the paper being an opinion piece, and thus not important, I would respectfully suggest that you are wrong. The author is John Ionnadis and he has published a number of meta-analyses demonstrating this problem. http://scholar.google.co.uk/scholar?hl=en&q=JPA+Ioannidis&btnG=Search&as_sdt=0%2C5&as_ylo=&as_vis=0 http://scholar.google.co.uk/scholar?hl=en&q=JPA+Ioannidi... Check out all the articles on the first page, he's been working in this area for a long time. The author of the piece also identifies a massive, massive problem with science. The problem is essentially, tenure is hard to get, and many people want it. Tenure is dependent on highly cited papers in good journals. These journals have a bias toward significant results which pushes scientists to data-dredge looking for any significant result. His system of pre-registration (which could be done on the ArXiV model, no government required) would allow us to have a much better idea of the things we're trying to study. My field (psychology, though I suspect the problem is just as bad in other areas) has a massive problem in this area, as a counter-intuitive result gets published in a good journal, while high quality replications which show no significant effects either don't get published or get published in a much lower ranking journal. This kind of registration becomes even more important when there are commercial interests riding on the outcome of a study, as in clinical trials. This (and the scandals) is presumably why such a system exists for clinical trials. Finally, I don't see how registering studies and their designs before running the tests creates a centralised approach to science. The research is all planned beforehand (for ethics committees at the very least), so there's no extra work involved. It also increases our trust in results, as the ones which make it through were predicted in advance, while the weird findings can be replicated. On a personal level, I can see the pressures to hypothesise after the results are known (HARK) and with tenure decisions looming, I can understand why people do it. Its horribly wrong, so this system would at least cut down on that behaviour. Full disclosure: almost every time I carry out research, the results are opposite to what I expect, so I am a somewhat biased participant in this debate.
- deleted 15y ago[deleted]