4 ms·

I suppose this is as good a place as any for this rant: If you go out seeking a specific type of data, it is almost impossible to find it. Multiple times in t

I suppose this is as good a place as any for this rant: If you go out seeking a specific type of data, it is almost impossible to find it. Multiple times in the past, I've wanted to sanity-check against public results, and gone looking for experimental data. I find plenty of papers, some of which even purport to measure the exact thing I'm interested in, but every single fucking one of them just has a plot. A god damn plot. Give me the data, I can make a god damn plot. I have MATLAB, I have gnuplot, it's no problem. But give me a plot, and I've got to fire up an image editor and count pixels - and of course it's a log-log plot, so I have to convert "pixels over" and "pixels up" to whatever units I actually need. I'm tired of this shit. I can't read half the papers I see because they're behind a paywall, and when I could read them, none of them included data. If you're an experimental scientist, your prose is probably worthless. I am wading through your entirely passive-voice description of your experimental setup (separate rant...) solely so that I can find some god damn data. Give me a drawing of your experimental setup, a csv of your data, and whatever MATLAB/LabView/Excel kludge you used for any derived quantities, and you can avoid something you obviously don't like (writing - or else you wouldn't do it so badly) and I can avoid something I don't like (reading your writing). Now that the internet exists, there's no reason not to communicate actual data sets - in the past, you'd have to deliver a phone book to your subscribers if you did that, so this all sort of makes sense in an anachronistic way, but it'd be so much more useful for the scientific community if the funding required "post your data set, drawings of your setup with specs for any instruments, and whatever MATLAB/LabView/Excel you used for any derived quantities" instead of "publish a short paper, it better have plots". (This doesn't really apply to multi-TB data sets like the author describes - obviously you're going to have to contact the PI and ask him/her to ship you a hard drive)

beoba 16y ago
Is there currently much motivation for a given researcher to keep the data online somewhere after the paper is out? Who would handle the hosting? Have you had any luck emailing the author of the paper?
- joe_the_user 16y ago
  Indeed, Everything from incentives to the attitude seems to make academic research groups act like sealed little islands of "brilliance". Researchers should be strongly "incentivized" to make their data and source code available and immediately usable (ie, not in Matlab). And the only way to do that is to gradually devalue data-less, source-code-less research in the way non-peer-review research is now devalued. (and yes, it will be messy, embarrassing, fragile code sometimes but we'll have to live with that). The amazing thing is that small, for-profit, private research organizations can still operate with this "what we produce is a pdf with a picture of our research on it" attitude. Btw, one organization can and I think is help is PLOS: http://www.plos.org/ http://www.plos.org/
  - neutronicus 16y ago
    > not in Matlab I disagree with this one. Matlab is nice, and I'm going to use it if it's available (and if I'm doing linear algebra - if the problem is more graph/tree-ish, I might reach for SciPy).
    - ylem 16y ago
      The problem with Matlab is that while it is fast for prototyping, it's difficult to write maintainable code in it (which is why I switched to python). Furthermore, you run into the issue of sharing code with people at institutions that don't have Matlab and Octave may or may not be compatible. Sometimes, an institution may have Matlab, but even if they do, they might not have access to the same toolkits you use (for example, I use a savitsky-golay filter as part of a peak finding algorithm--because it's in Matlab and we have a DSP toolkit, I just used that one. Because my collaborators didn't have it, I ended up rewriting the code in python--and this was for a group at a national lab...Now, imagine people at small universities and colleges...)
      - neutronicus 16y ago
        I use Matlab mostly as a souped-up excel, for prototyping, and for the parallel computing toolbox. If I plan on maintaining something, I want static typing, so the whole SciPy/Matlab/Octave/R debate is a little moot for me there.
Create 16y ago
oh, come on. If you have done real science, you should know, that asking for scientific data is like asking for the source code of Windows. You have valid points (it has been published, you have licensed access, you can do cool stuff with it, verify and iron out the bugs etc.): but this is not the way it works. Reality does not match the sharing knowledge mantra - not to mention fudge factors and data context (experiment design, knowledge of measuring device, detector etc.). LEP data is not only useless because it is old and accessible via the dinosaurish FORTRAN, but because the knowledge of the metadata (i.e. shower profile of an endcap) almost gone and certainly not available to outsiders. Reproducible experiments themselves are becoming the exception and not the norm. In every field. (not to mention specialisation).
Someone 16y ago
Now that the internet exists, there's no reason not to communicate actual data sets That is not always true. For many sets involving humans, data may have been collected under the express promise tat anything published will not be used for anything else and/or cannot be de-anonymized. Verifying that that is the case can take quite an effort, especially when considering that third parties can combine raw data from multiple papers building on the same data.

beoba 16y ago

Is there currently much motivation for a given researcher to keep the data online somewhere after the paper is out? Who would handle the hosting? Have you had any luck emailing the author of the paper?

joe_the_user 16y ago
Indeed, Everything from incentives to the attitude seems to make academic research groups act like sealed little islands of "brilliance". Researchers should be strongly "incentivized" to make their data and source code available and immediately usable (ie, not in Matlab). And the only way to do that is to gradually devalue data-less, source-code-less research in the way non-peer-review research is now devalued. (and yes, it will be messy, embarrassing, fragile code sometimes but we'll have to live with that). The amazing thing is that small, for-profit, private research organizations can still operate with this "what we produce is a pdf with a picture of our research on it" attitude. Btw, one organization can and I think is help is PLOS: http://www.plos.org/ http://www.plos.org/
- neutronicus 16y ago
  > not in Matlab I disagree with this one. Matlab is nice, and I'm going to use it if it's available (and if I'm doing linear algebra - if the problem is more graph/tree-ish, I might reach for SciPy).
  - ylem 16y ago
    The problem with Matlab is that while it is fast for prototyping, it's difficult to write maintainable code in it (which is why I switched to python). Furthermore, you run into the issue of sharing code with people at institutions that don't have Matlab and Octave may or may not be compatible. Sometimes, an institution may have Matlab, but even if they do, they might not have access to the same toolkits you use (for example, I use a savitsky-golay filter as part of a peak finding algorithm--because it's in Matlab and we have a DSP toolkit, I just used that one. Because my collaborators didn't have it, I ended up rewriting the code in python--and this was for a group at a national lab...Now, imagine people at small universities and colleges...)
    - neutronicus 16y ago
      I use Matlab mostly as a souped-up excel, for prototyping, and for the parallel computing toolbox. If I plan on maintaining something, I want static typing, so the whole SciPy/Matlab/Octave/R debate is a little moot for me there.

Create 16y ago

oh, come on. If you have done real science, you should know, that asking for scientific data is like asking for the source code of Windows. You have valid points (it has been published, you have licensed access, you can do cool stuff with it, verify and iron out the bugs etc.): but this is not the way it works. Reality does not match the sharing knowledge mantra - not to mention fudge factors and data context (experiment design, knowledge of measuring device, detector etc.). LEP data is not only useless because it is old and accessible via the dinosaurish FORTRAN, but because the knowledge of the metadata (i.e. shower profile of an endcap) almost gone and certainly not available to outsiders. Reproducible experiments themselves are becoming the exception and not the norm. In every field. (not to mention specialisation).

Someone 16y ago

Now that the internet exists, there's no reason not to communicate actual data sets That is not always true. For many sets involving humans, data may have been collected under the express promise tat anything published will not be used for anything else and/or cannot be de-anonymized. Verifying that that is the case can take quite an effort, especially when considering that third parties can combine raw data from multiple papers building on the same data.