4 ms·

This is one of those things that makes me deeply skeptical of the most zealous folks within the open science / open access movements. The emphasis seems to be

This is one of those things that makes me deeply skeptical of the most zealous folks within the open science / open access movements. The emphasis seems to be much more on just throwing your data / paper up on the web sans context and much less on properly adding metadata and documentation for how to perform an experiment or how data can be reused. I've probably said this before elsewhere, but Josh Carp's "The Secret Lives of fMRI Experiments" really underscores this point. There are two equally catastrophic issues in the social (and to an extent) natural sciences and solving one (openness) is only part of the puzzle when the other involves blatant sloppiness in bookkeeping, pure and simple. If science is to overcome its current setbacks and lots of confidence, it needs to emphasize rigor alongside transparency.

traverseda 9y ago
All the metadata stuff is good, but I'm not in academia. I can't make use of it if it's not available. That's why I'm worried about the information being available first and foremost.
- neuromantik8086 9y ago
  > I can't make use of it if it's not available You also can't make use of data if it doesn't have appropriate metadata indicating how it can be used. Metadata is more than just "good"- it's essential for actually being able to run an analysis. Take fMRI for example- someone could just throw a brain image up on the internet. In fact, one particularly uninformed individual (who possibly broke Virginia state law) did so here: https://github.com/dcunited001/mri-scans https://github.com/dcunited001/mri-scans However, the raw data is not useful without knowing what scan sequence was used, slice-timing order, etc. Some of this is encoded in the data itself (the example above uploaded DICOM, which has somewhat rich metadata, although by no means is it comprehensive enough), but other data that is necessary is not (Phillips in particular is somewhat infamous for doing non-standard things with their scanners that aren't recorded in the DICOM headers, and in general it's a Herculean task getting MRI vendors to agree on standards). You could try performing an analysis with contextless data, but you're likely just going to make really spotty conclusions. > I'm not in academia Why should that matter? The circumstances of data collection are also important for business intelligence data or financial modeling data. I'm certain that the lack of this also contributes to poorly formed conclusions that cause real hurt to businesses out there.
  - neuromantik8086 9y ago
    Nevermind- I see where you're coming from now wrt not being in academia and not having access to data at all. My point still stands though- I think that you're grouping metadata and data together when you refer to "data".
jononor 9y ago
The publishing of data and code openly is just the bare minimum. But from that we can start to build more reproducable science.

traverseda 9y ago

All the metadata stuff is good, but I'm not in academia. I can't make use of it if it's not available. That's why I'm worried about the information being available first and foremost.

neuromantik8086 9y ago
> I can't make use of it if it's not available You also can't make use of data if it doesn't have appropriate metadata indicating how it can be used. Metadata is more than just "good"- it's essential for actually being able to run an analysis. Take fMRI for example- someone could just throw a brain image up on the internet. In fact, one particularly uninformed individual (who possibly broke Virginia state law) did so here: https://github.com/dcunited001/mri-scans https://github.com/dcunited001/mri-scans However, the raw data is not useful without knowing what scan sequence was used, slice-timing order, etc. Some of this is encoded in the data itself (the example above uploaded DICOM, which has somewhat rich metadata, although by no means is it comprehensive enough), but other data that is necessary is not (Phillips in particular is somewhat infamous for doing non-standard things with their scanners that aren't recorded in the DICOM headers, and in general it's a Herculean task getting MRI vendors to agree on standards). You could try performing an analysis with contextless data, but you're likely just going to make really spotty conclusions. > I'm not in academia Why should that matter? The circumstances of data collection are also important for business intelligence data or financial modeling data. I'm certain that the lack of this also contributes to poorly formed conclusions that cause real hurt to businesses out there.
- neuromantik8086 9y ago
  Nevermind- I see where you're coming from now wrt not being in academia and not having access to data at all. My point still stands though- I think that you're grouping metadata and data together when you refer to "data".

jononor 9y ago

The publishing of data and code openly is just the bare minimum. But from that we can start to build more reproducable science.