3 ms·

Jelte Wicherts and his co-authors put a set of general suggestions for more open data in science research in an article in Frontiers of Computational Neuroscien

Jelte Wicherts and his co-authors put a set of general suggestions for more open data in science research in an article in Frontiers of Computational Neuroscience (an open-access journal).[1] "With the emergence of online publishing, opportunities to maximize transparency of scientific research have grown considerably. However, these possibilities are still only marginally used. We argue for the implementation of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and (3) online data publication. First, peer-reviewed peer review entails a community-wide review system in which reviews are published online and rated by peers. This ensures accountability of reviewers, thereby increasing academic quality of reviews. Second, reviewers who write many highly regarded reviews may move to higher editorial positions. Third, online publication of data ensures the possibility of independent verification of inferential claims in published papers. This counters statistical errors and overly positive reporting of statistical results. We illustrate the benefits of these strategies by discussing an example in which the classical publication system has gone awry, namely controversial IQ research. We argue that this case would have likely been avoided using more transparent publication practices. We argue that the proposed system leads to better reviews, meritocratic editorial hierarchies, and a higher degree of replicability of statistical analyses." Wicherts has published another article, "Publish (Your Data) or (Let the Data) Perish! Why Not Publish Your Data Too?"[2] on how important it is to make data available to other researchers. Wicherts does a lot of research on this issue to try to reduce the number of dubious publications in his main discipline, the psychology of human intelligence. When I see a new publication of primary research in that discipline, I don't take it seriously at all as a description of the facts of the world until I have read that independent researchers have examined the first author's data and found that they check out. Often the data are unavailable, or were misanalyzed in the first place. [1] Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker and Denny Borsboom. Letting the daylight in: reviewing the reviewers and other ways to maximize transparency in science. Front. Comput. Neurosci., 03 April 2012 doi: 10.3389/fncom.2012.00020 http://www.frontiersin.org/Computational_Neuroscience/10.3389/fncom.2012.00020/full http://www.frontiersin.org/Computational_Neuroscience/10.338... [2] Wicherts, J.M. & Bakker, M. (2012). Publish (your data) or (let the data) perish! Why not publish your data too? Intelligence,40, 73-76. http://wicherts.socsci.uva.nl/Wichertsbakker2012.pdf http://wicherts.socsci.uva.nl/Wichertsbakker2012.pdf

omegant 13y ago
It's an amazing topic. My wife is a doctor and I'm amazed of how difficult is to see if an article is sound or important or it's been checked properly, or if the tree of citations that lead to it's importance is sound after new studies. I've been thinking on a visual standard that let's you know what status has a given study, how many reviews has, if it's citing studies that are solid with solid peer reviews and such. Ideally it should be possible to track a new discovery down to it's scientific roots, and easily be able to know how good is. But to achieve this you need as you said to change all the scientific publication method, create some kind of standard publication API for the data, the citations, the peer reviews and number of times it's been replicated. Also some kind of debate forum around this publications would be of help. Were people could coordinate replications or talk about methods , etc... But it is all a big mess, and certainly ready for disruption. One clear user case is that recently it was published in the New England (a tier one med publication) how all the studies about high blood pressure and salt relationship have their origin in an old animal study with rabbits (If I recall correctly). They fed them with the equivalent for humans of hundreds of grams of salt, as the blood pressure rise it was deduced that salt causes high blood pressure. They also did a meta-study about the relationship of high blood pressure and salt and they didn't find a clear correlation. Maybe this is correct maybe it´s not, but that a medical truth as established as salt=high blood pressure can not be properly traced and known, just let's you see how it's all broken. edit: typos and spelling

omegant 13y ago

It's an amazing topic. My wife is a doctor and I'm amazed of how difficult is to see if an article is sound or important or it's been checked properly, or if the tree of citations that lead to it's importance is sound after new studies. I've been thinking on a visual standard that let's you know what status has a given study, how many reviews has, if it's citing studies that are solid with solid peer reviews and such. Ideally it should be possible to track a new discovery down to it's scientific roots, and easily be able to know how good is. But to achieve this you need as you said to change all the scientific publication method, create some kind of standard publication API for the data, the citations, the peer reviews and number of times it's been replicated. Also some kind of debate forum around this publications would be of help. Were people could coordinate replications or talk about methods , etc... But it is all a big mess, and certainly ready for disruption. One clear user case is that recently it was published in the New England (a tier one med publication) how all the studies about high blood pressure and salt relationship have their origin in an old animal study with rabbits (If I recall correctly). They fed them with the equivalent for humans of hundreds of grams of salt, as the blood pressure rise it was deduced that salt causes high blood pressure. They also did a meta-study about the relationship of high blood pressure and salt and they didn't find a clear correlation. Maybe this is correct maybe it´s not, but that a medical truth as established as salt=high blood pressure can not be properly traced and known, just let's you see how it's all broken. edit: typos and spelling