6 ms·

Is there currently much motivation for a given researcher to keep the data online somewhere after the paper is out? Who would handle the hosting? Have you had

by beoba 16y ago

Is there currently much motivation for a given researcher to keep the data online somewhere after the paper is out? Who would handle the hosting? Have you had any luck emailing the author of the paper?

joe_the_user 16y ago
Indeed, Everything from incentives to the attitude seems to make academic research groups act like sealed little islands of "brilliance". Researchers should be strongly "incentivized" to make their data and source code available and immediately usable (ie, not in Matlab). And the only way to do that is to gradually devalue data-less, source-code-less research in the way non-peer-review research is now devalued. (and yes, it will be messy, embarrassing, fragile code sometimes but we'll have to live with that). The amazing thing is that small, for-profit, private research organizations can still operate with this "what we produce is a pdf with a picture of our research on it" attitude. Btw, one organization can and I think is help is PLOS: http://www.plos.org/ http://www.plos.org/
- neutronicus 16y ago
  > not in Matlab I disagree with this one. Matlab is nice, and I'm going to use it if it's available (and if I'm doing linear algebra - if the problem is more graph/tree-ish, I might reach for SciPy).
  - ylem 16y ago
    The problem with Matlab is that while it is fast for prototyping, it's difficult to write maintainable code in it (which is why I switched to python). Furthermore, you run into the issue of sharing code with people at institutions that don't have Matlab and Octave may or may not be compatible. Sometimes, an institution may have Matlab, but even if they do, they might not have access to the same toolkits you use (for example, I use a savitsky-golay filter as part of a peak finding algorithm--because it's in Matlab and we have a DSP toolkit, I just used that one. Because my collaborators didn't have it, I ended up rewriting the code in python--and this was for a group at a national lab...Now, imagine people at small universities and colleges...)
    - neutronicus 16y ago
      I use Matlab mostly as a souped-up excel, for prototyping, and for the parallel computing toolbox. If I plan on maintaining something, I want static typing, so the whole SciPy/Matlab/Octave/R debate is a little moot for me there.
      - ylem 16y ago
        I don't want to start a flame war, but I think static typing is overrated.
        neutronicus 16y ago
        I like being able to see function signatures in a header file. In Python or Matlab, I have to read the implementation to see what types it wants. To me, having collections of type signatures in a separate file is a huge boon when I have to come back to code I haven't seen for a while - it's a 10,000 foot view of what the code does that helps jog my memory - and it's guaranteed to be accurate, unlike documentation. To the extent that static typing supposedly prevents bugs, I agree that it is overrated. My neutron transport solver I wrote in Haskell had plenty of god damn bugs after I got it to compile, thank you very much (perhaps a very deep Haskell-fu is required to attain the mythical "neutron transport solver that works right the first time" - in any case I realized that compiler-aided bug prevention is not a low-hanging fruit in Haskell, so I lost interest). That was a long-winded way to say that I feel lost and alone without function signatures, and that this preference is possibly only quasi-rational.
hammerdr 16y ago
> Who would handle the hosting? The university and/or the journal. Supposedly, part of their existence and social contract is to maintain knowledge archives for public benefit (even if its not free).
jules 16y ago
University members, even students, nearly always get free hosting space. The problem is not the cost of hosting, or even the time it takes to put it up. The problem is the competition between research groups. If you publish your data, the competition can use it to beat you at the next discovery in the often narrow field.
neutronicus 16y ago
> Have you had any luck emailing the author of the paper? This works sometimes. People want to be cited after all. Sometimes you just never get a response. Also not viable for lab courses in graduate school, where you need to "compare your results with the literature", or for "calibrate my instruments" type situations (rather than "cite you" type situations) - there's some data that would be useful for things that aren't really worth bothering a PI over. If I were a funding agency, I would earmark funding specifically for hosting data, because that's the biggest value of a lot of experimental science. A lot of times the data analysis is pedestrian, but actually taking measurements is difficult and expensive.
Create 16y ago
not much motivation: most research is done on a "works for me" and "good enough, got the 5 sigma" level. Reproducible industrial ISO protocols to which you would need to add user support is certainly not granted for in grant applications. Not to mention data formats, software versions and the number of flies in the lab.
- ylem 16y ago
  It's an incentive question: Researchers are often evaluated based on the papers they produce (impact and numbers)--not based on how maintainable our code base is. If I can keep a cobbled together code base going and still produce code, then it's in my best interest to do science rather than write recyclable code. The other problem that happens is that some people don't release their source code (they may release binaries) because either they're embarrassed by it, or because they want to maintain competitive advantage....

by beoba 16y ago

joe_the_user 16y ago

Indeed, Everything from incentives to the attitude seems to make academic research groups act like sealed little islands of "brilliance". Researchers should be strongly "incentivized" to make their data and source code available and immediately usable (ie, not in Matlab). And the only way to do that is to gradually devalue data-less, source-code-less research in the way non-peer-review research is now devalued. (and yes, it will be messy, embarrassing, fragile code sometimes but we'll have to live with that). The amazing thing is that small, for-profit, private research organizations can still operate with this "what we produce is a pdf with a picture of our research on it" attitude. Btw, one organization can and I think is help is PLOS: http://www.plos.org/ http://www.plos.org/

neutronicus 16y ago
> not in Matlab I disagree with this one. Matlab is nice, and I'm going to use it if it's available (and if I'm doing linear algebra - if the problem is more graph/tree-ish, I might reach for SciPy).
- ylem 16y ago
  The problem with Matlab is that while it is fast for prototyping, it's difficult to write maintainable code in it (which is why I switched to python). Furthermore, you run into the issue of sharing code with people at institutions that don't have Matlab and Octave may or may not be compatible. Sometimes, an institution may have Matlab, but even if they do, they might not have access to the same toolkits you use (for example, I use a savitsky-golay filter as part of a peak finding algorithm--because it's in Matlab and we have a DSP toolkit, I just used that one. Because my collaborators didn't have it, I ended up rewriting the code in python--and this was for a group at a national lab...Now, imagine people at small universities and colleges...)
  - neutronicus 16y ago
    I use Matlab mostly as a souped-up excel, for prototyping, and for the parallel computing toolbox. If I plan on maintaining something, I want static typing, so the whole SciPy/Matlab/Octave/R debate is a little moot for me there.
    - ylem 16y ago
      I don't want to start a flame war, but I think static typing is overrated.
      - neutronicus 16y ago
        I like being able to see function signatures in a header file. In Python or Matlab, I have to read the implementation to see what types it wants. To me, having collections of type signatures in a separate file is a huge boon when I have to come back to code I haven't seen for a while - it's a 10,000 foot view of what the code does that helps jog my memory - and it's guaranteed to be accurate, unlike documentation. To the extent that static typing supposedly prevents bugs, I agree that it is overrated. My neutron transport solver I wrote in Haskell had plenty of god damn bugs after I got it to compile, thank you very much (perhaps a very deep Haskell-fu is required to attain the mythical "neutron transport solver that works right the first time" - in any case I realized that compiler-aided bug prevention is not a low-hanging fruit in Haskell, so I lost interest). That was a long-winded way to say that I feel lost and alone without function signatures, and that this preference is possibly only quasi-rational.

hammerdr 16y ago

> Who would handle the hosting? The university and/or the journal. Supposedly, part of their existence and social contract is to maintain knowledge archives for public benefit (even if its not free).

jules 16y ago

University members, even students, nearly always get free hosting space. The problem is not the cost of hosting, or even the time it takes to put it up. The problem is the competition between research groups. If you publish your data, the competition can use it to beat you at the next discovery in the often narrow field.

neutronicus 16y ago

> Have you had any luck emailing the author of the paper? This works sometimes. People want to be cited after all. Sometimes you just never get a response. Also not viable for lab courses in graduate school, where you need to "compare your results with the literature", or for "calibrate my instruments" type situations (rather than "cite you" type situations) - there's some data that would be useful for things that aren't really worth bothering a PI over. If I were a funding agency, I would earmark funding specifically for hosting data, because that's the biggest value of a lot of experimental science. A lot of times the data analysis is pedestrian, but actually taking measurements is difficult and expensive.

Create 16y ago

not much motivation: most research is done on a "works for me" and "good enough, got the 5 sigma" level. Reproducible industrial ISO protocols to which you would need to add user support is certainly not granted for in grant applications. Not to mention data formats, software versions and the number of flies in the lab.

ylem 16y ago
It's an incentive question: Researchers are often evaluated based on the papers they produce (impact and numbers)--not based on how maintainable our code base is. If I can keep a cobbled together code base going and still produce code, then it's in my best interest to do science rather than write recyclable code. The other problem that happens is that some people don't release their source code (they may release binaries) because either they're embarrassed by it, or because they want to maintain competitive advantage....