6 ms·
I think there's also a lot of confusion about what the scientific literature should be. Outsiders think of journals as pre-packaged nuggets of science fact, wi
by epistasis 10y ago
I think there's also a lot of confusion about what the scientific literature should be.
Outsiders think of journals as pre-packaged nuggets of science fact, with conclusions that you can read and trust.
Scientists who publish in the journals view them as a way to communicate a body of work of "I did this, here is the result I saw. I think it might mean this."
The difference between these two views can not be understated. Each group wants the literature to be useful to them, so that they can use it, and that's understandable.
For most areas of science, especially in the early days of that science, it's absolutely essential that scientist's view be allowed to persist. Is it better to share early, or to wait to publish until you've tried all possible things that could potentially go wrong? I think the answer is obviously that you share the early data, and what you think it means, even if you may be wrong about it.
If the goal is to advance knowledge as quickly as possible, I think that the scientist's view is probably a better idea of what a journal needs to be than what the outsiders' view is. In some fields, like fMRI studies, the field is realizing that they may need to go about things differently. And that means that a lot of the interpretations that were published earlier are incorrect. But that process of incorrect interpretation to corrected interpretation is an essential part of science.
- panic 10y agoScientists and academic institutions add to the confusion with the way they promote their work. Nobody puts language like "I did this, here is the result I saw. I think it might mean this" in a press release. It's always "Breakthrough research! Is the end of cancer in sight??"
- epistasis 10y agoAbsolutely true, though I know of very few scientists that have been able to tamp down the game of telephone that the university's press office starts. http://www.phdcomics.com/comics.php?n=1174 http://www.phdcomics.com/comics.php?n=1174
- nostrademons 10y agoA lot of this is because of the incentives that result from grant-based funding of science. Your ability to continue to do science is contingent on your ability to find someone to pay for it; it's much easier to convince someone to pay for it if you can point to headlines that say "Breakthrough research! Is the end of cancer in sight?" The old system of "gentleman science" (where only independently-wealthy heirs did science, as a hobby) had a lot of problems and wouldn't really be workable today, but one thing it had going for it was that the scientist could count on the funding being there tomorrow, and so had much less of an incentive to overstate their work.
- averagewall 10y agoIt doesn't help when teachers tell their students they can reference academic papers from peer reviewed journals but can't reference Wikipedia. The former is tentative and the latter is established knowledge from text books, more or less.
- ivan_gammel 10y agoProbably, the real problem is that "the scientific literature should be". It's not the right question to discuss today, because scientific literature is just a brief approximation of scientific knowledge and this is not what we need. Whole modern scientific system is outdated and has to be upgraded: journals, citation indexes, degrees should go away and be replaced. Science should become more formal and more digital, so that it will be easier to find and validate the results. There should be databases, search engines and digital signatures of the raw data and chains of proofs. Counting citations should be replaced with counting chains of proof produced by the scientist and included in others' works. Reliability of proof and contribution to ratings should be based on independent confirmations, which should be counted too - replication of new results should be considered an achievement too. Journals may remain, but not as a primary way to exchange scientific information - more likely, they should become the portals to the data which will link the results in databases to their human-readable interpretations.
- entee 10y agoRespectfully, I think you're missing the complications outlined in the rest of the thread. The difficulty in the biosciences and even in chemistry and some physics is that they can't really be formalized in the way that is necessary to construct these "chains of proof". The scientific literature is an ongoing conversation anchored by rigorous experimental facts and data. But rigorous doesn't mean it's clean like a mathematical proof. In fact, most science approaches its "proofs" in quite a different way than math. For example, as far as science would be concerned P!=NP in complexity theory. We've done the experiment many times, tried different things, it's pretty much true. But it's still not mathematically proven because there isn't a formal proof. That's not to say it's invalid to expect more rigor, or that we wouldn't all love to have "chains of proof" and databases and signatures for data etc. It's that it's simply not practical given how noisy and complex biological systems are. In contrast to math, you pretty much never know the full complement of objects/chemicals/parameters in your experimental space. You try to do the right controls to eliminate the confounding variables, but you're still never fully in control of all the nobs and switches in your system. That's why usually you need multiple different experiments tackling a problem from multiple different approaches for a result to be convincing. Formalized systems would be great, but I don't think we're even close to understanding how to properly formalize all of those difficulties and variables in a useful way. And it may not even be possible.