4 ms·
The argument against this, and I make this as a jaded and struggling graduate student so accept that bias, is that there is no acceptable outlet for negative re
by lanaius 13y ago
The argument against this, and I make this as a jaded and struggling graduate student so accept that bias, is that there is no acceptable outlet for negative results. There is also no acceptable outlet for those who consistently produce negative results and so, as I'm sure everyone understands, there is tremendous pressure to produce positive and interesting results. We hear about the times that this is forced due to manipulation and falsification and this is never okay, but there are a great many times where it falls into a nebulous gray area. We double and triple check our code, verify simulation results, and run down the checklists but that never guarantees that we do not have such "excel column errors".
If data and code are open this is fantastic for science and progress because errors do not replicate and permute but it can be terrible for individual scientists and graduate students. As always, it comes down to money. If we are forced to correct or even worse retract a scientific publication, the community (both local and worldwide) bares down on you like a knife. The stakes are so high that not only is falsification lethal, in many cases honest mistakes can be lethal as well. Conversely one cannot take eternity flipping every stone to bulletproof every possible single problem and it can be very hard to identify which stones to check!
This is not a defense of closed work and closed data, but it's a realization that opening data and code is not a simple and straightforward process. There are severe and deeply embedded cultural problems that make post-publication sharing difficult.
- pseut 13y agoI don't know your field, but the claim that there's no acceptable outlet for negative results isn't always true. There are usually outlets for interesting results, it's just that negative results tend to be less interesting than positive ones. Negative results that blow up accepted wisdom, though... An example from economics: Meese and Rogoff [1] has (by google scholar) about 3000 citations, which is massive for econ and is one of the key references in Intl Macro. They showed that all of the models in use at the time of publication sucked. [1] Empirical exchange rate models of the seventies: do they fit out of sample? links to the paper here: http://scholar.google.com/scholar?cluster=17091735262456791337&hl=en&as_sdt=0,16 http://scholar.google.com/scholar?cluster=170917352624567913...