3 ms·

But there should also be replication of the concepts of the study under a different paradigm, otherwise you might just be finding out effects of the particular

by bbctol 10y ago

But there should also be replication of the concepts of the study under a different paradigm, otherwise you might just be finding out effects of the particular procedure. An original study is rarely "wrong" (fraud is a serious problem, but not a typical one), it's that it's merely one data point, from one procedure, and a lot of the time the effect that can be replicated is "real," it's just a side effect of the process. Basically, there's way more replication that should be going on in science at many levels.

jerf 10y ago
Yes. But replication is a starting point. It isn't just about fraud, it's also about error and failure to communicate the experiment. (It's also beyond time for paper journals to just contain the relevant summaries and conclusions for given studies and for "what fits in a journal" to cease being "what fits in publication". Authors ought to be able to publish as much stuff as they want with a paper, including full source code sets or data sets or whatever, with the paper limitations affecting only the top-level summary.)
- bbctol 10y ago
  So I work for a scientific journal (insert comments about how evil I am here) and we have much bigger problems with authors not providing full data than with us refusing it. It is almost always possible to upload a full data set and code to a paper; what we're working on now is standardizing formats as much as possible so it's easy to immediately see what equipment and reagents were used, where the experimental procedures came from, what data was gathered and when, and how exactly it was analyzed without grabbing tons of files in different formats. "As much information as authors want" is high for an excellent but small set of authors who are probably right that the journal should do more to facilitate hosting their data--for most authors, that's far less information than we actually need.
  - jerf 10y ago
    Yes, I do not mean to just criticize journals, authors also need to start realizing they need to produce that as well. 10-20ish pages of paper for the work of a dozen people for three years (and that for the good-to-great papers) is just not an acceptable payoff for the amount of money involved.

by bbctol 10y ago

jerf 10y ago

Yes. But replication is a starting point. It isn't just about fraud, it's also about error and failure to communicate the experiment. (It's also beyond time for paper journals to just contain the relevant summaries and conclusions for given studies and for "what fits in a journal" to cease being "what fits in publication". Authors ought to be able to publish as much stuff as they want with a paper, including full source code sets or data sets or whatever, with the paper limitations affecting only the top-level summary.)

bbctol 10y ago
So I work for a scientific journal (insert comments about how evil I am here) and we have much bigger problems with authors not providing full data than with us refusing it. It is almost always possible to upload a full data set and code to a paper; what we're working on now is standardizing formats as much as possible so it's easy to immediately see what equipment and reagents were used, where the experimental procedures came from, what data was gathered and when, and how exactly it was analyzed without grabbing tons of files in different formats. "As much information as authors want" is high for an excellent but small set of authors who are probably right that the journal should do more to facilitate hosting their data--for most authors, that's far less information than we actually need.
- jerf 10y ago
  Yes, I do not mean to just criticize journals, authors also need to start realizing they need to produce that as well. 10-20ish pages of paper for the work of a dozen people for three years (and that for the good-to-great papers) is just not an acceptable payoff for the amount of money involved.