7 ms·

Dumped on by Data: Scientists Say a Deluge is Drowning Research

33 points by grellas 16y ago

zipdog 16y ago
It seems like there needs to be a real break between the collectors of data and the analyzers. At the moment heaps of data is collected by the same teams that analyze it, but just as an API opens up a codebase to new ideas, so would a standard practice of placing the data into a library before starting analysis. Fortunately there's already a growing number of places (typically government in my experience) making an effort just to keep data libraries up and open to further research.
- fauigerzigerk 16y ago
  I agree that this is a problem, but I fear it's not as easy to solve as with traditional libraries that store books. For data to be useful you need a lot of very formal high quality metadata, and that metadata has to be collected by the original researchers as part of their process. So I think there has to be an incentive or mandate that makes researchers collect and document data in a more systematic way. Librarians cannot do it for them.
tgflynn 16y ago
Why are article titles on the web so disconnected from the contents of the posts ? This title to me meant "Scientists are complaining that greater availability of data is making it harder to do research." Instead it turns out the post is about scientists losing data due to not having places/formats for storing it. I have trouble understanding how someone thought this was a good title for this post. Are titles being chosen simply to maximize click rates ?
- Alex3917 16y ago
  "Are titles being chosen simply to maximize click rates?" I was under the impression that the editors usually choose four or five different titles, and then after the first few thousand impressions the software chooses the one that does the best to become the permanent title. Apropos of this, a cool hack is that the more days in a row you click on an item in Amazon the cheaper it gets. So if you want to save an extra 10% on something, just click it once a day for a week or so until the price goes down. (After a certain point though it starts going back up again, so you have to time it right.)
  - beoba 16y ago
    Straying a bit from the original topic: For amazon price tracking I've used this and it's pretty good: http://camelcamelcamel.com/ http://camelcamelcamel.com/ (They apparently cover some other retailers too.)
    - L1quid 16y ago
      Glad you like it (I'm the author.)
neutronicus 16y ago
I suppose this is as good a place as any for this rant: If you go out seeking a specific type of data, it is almost impossible to find it. Multiple times in the past, I've wanted to sanity-check against public results, and gone looking for experimental data. I find plenty of papers, some of which even purport to measure the exact thing I'm interested in, but every single fucking one of them just has a plot. A god damn plot. Give me the data, I can make a god damn plot. I have MATLAB, I have gnuplot, it's no problem. But give me a plot, and I've got to fire up an image editor and count pixels - and of course it's a log-log plot, so I have to convert "pixels over" and "pixels up" to whatever units I actually need. I'm tired of this shit. I can't read half the papers I see because they're behind a paywall, and when I could read them, none of them included data. If you're an experimental scientist, your prose is probably worthless. I am wading through your entirely passive-voice description of your experimental setup (separate rant...) solely so that I can find some god damn data. Give me a drawing of your experimental setup, a csv of your data, and whatever MATLAB/LabView/Excel kludge you used for any derived quantities, and you can avoid something you obviously don't like (writing - or else you wouldn't do it so badly) and I can avoid something I don't like (reading your writing). Now that the internet exists, there's no reason not to communicate actual data sets - in the past, you'd have to deliver a phone book to your subscribers if you did that, so this all sort of makes sense in an anachronistic way, but it'd be so much more useful for the scientific community if the funding required "post your data set, drawings of your setup with specs for any instruments, and whatever MATLAB/LabView/Excel you used for any derived quantities" instead of "publish a short paper, it better have plots". (This doesn't really apply to multi-TB data sets like the author describes - obviously you're going to have to contact the PI and ask him/her to ship you a hard drive)
- beoba 16y ago
  Is there currently much motivation for a given researcher to keep the data online somewhere after the paper is out? Who would handle the hosting? Have you had any luck emailing the author of the paper?
  - joe_the_user 16y ago
    Indeed, Everything from incentives to the attitude seems to make academic research groups act like sealed little islands of "brilliance". Researchers should be strongly "incentivized" to make their data and source code available and immediately usable (ie, not in Matlab). And the only way to do that is to gradually devalue data-less, source-code-less research in the way non-peer-review research is now devalued. (and yes, it will be messy, embarrassing, fragile code sometimes but we'll have to live with that). The amazing thing is that small, for-profit, private research organizations can still operate with this "what we produce is a pdf with a picture of our research on it" attitude. Btw, one organization can and I think is help is PLOS: http://www.plos.org/ http://www.plos.org/
robbles 16y ago
sounds like a good application for Fluidinfo: http://www.fluidinfo.com/about/ http://www.fluidinfo.com/about/ if they made it a bit more accessible, scientists could potentially share some of their data, and new research could associate their related findings with the original data.

33 points by grellas 16y ago

It seems like there needs to be a real break between the collectors of data and the analyzers. At the moment heaps of data is collected by the same teams that analyze it, but just as an API opens up a codebase to new ideas, so would a standard practice of placing the data into a library before starting analysis. Fortunately there's already a growing number of places (typically government in my experience) making an effort just to keep data libraries up and open to further research.

fauigerzigerk 16y ago
I agree that this is a problem, but I fear it's not as easy to solve as with traditional libraries that store books. For data to be useful you need a lot of very formal high quality metadata, and that metadata has to be collected by the original researchers as part of their process. So I think there has to be an incentive or mandate that makes researchers collect and document data in a more systematic way. Librarians cannot do it for them.

tgflynn 16y ago

Why are article titles on the web so disconnected from the contents of the posts ? This title to me meant "Scientists are complaining that greater availability of data is making it harder to do research." Instead it turns out the post is about scientists losing data due to not having places/formats for storing it. I have trouble understanding how someone thought this was a good title for this post. Are titles being chosen simply to maximize click rates ?

Alex3917 16y ago
"Are titles being chosen simply to maximize click rates?" I was under the impression that the editors usually choose four or five different titles, and then after the first few thousand impressions the software chooses the one that does the best to become the permanent title. Apropos of this, a cool hack is that the more days in a row you click on an item in Amazon the cheaper it gets. So if you want to save an extra 10% on something, just click it once a day for a week or so until the price goes down. (After a certain point though it starts going back up again, so you have to time it right.)
- beoba 16y ago
  Straying a bit from the original topic: For amazon price tracking I've used this and it's pretty good: http://camelcamelcamel.com/ http://camelcamelcamel.com/ (They apparently cover some other retailers too.)
  - L1quid 16y ago
    Glad you like it (I'm the author.)

neutronicus 16y ago

I suppose this is as good a place as any for this rant: If you go out seeking a specific type of data, it is almost impossible to find it. Multiple times in the past, I've wanted to sanity-check against public results, and gone looking for experimental data. I find plenty of papers, some of which even purport to measure the exact thing I'm interested in, but every single fucking one of them just has a plot. A god damn plot. Give me the data, I can make a god damn plot. I have MATLAB, I have gnuplot, it's no problem. But give me a plot, and I've got to fire up an image editor and count pixels - and of course it's a log-log plot, so I have to convert "pixels over" and "pixels up" to whatever units I actually need. I'm tired of this shit. I can't read half the papers I see because they're behind a paywall, and when I could read them, none of them included data. If you're an experimental scientist, your prose is probably worthless. I am wading through your entirely passive-voice description of your experimental setup (separate rant...) solely so that I can find some god damn data. Give me a drawing of your experimental setup, a csv of your data, and whatever MATLAB/LabView/Excel kludge you used for any derived quantities, and you can avoid something you obviously don't like (writing - or else you wouldn't do it so badly) and I can avoid something I don't like (reading your writing). Now that the internet exists, there's no reason not to communicate actual data sets - in the past, you'd have to deliver a phone book to your subscribers if you did that, so this all sort of makes sense in an anachronistic way, but it'd be so much more useful for the scientific community if the funding required "post your data set, drawings of your setup with specs for any instruments, and whatever MATLAB/LabView/Excel you used for any derived quantities" instead of "publish a short paper, it better have plots". (This doesn't really apply to multi-TB data sets like the author describes - obviously you're going to have to contact the PI and ask him/her to ship you a hard drive)

beoba 16y ago
Is there currently much motivation for a given researcher to keep the data online somewhere after the paper is out? Who would handle the hosting? Have you had any luck emailing the author of the paper?
- joe_the_user 16y ago
  Indeed, Everything from incentives to the attitude seems to make academic research groups act like sealed little islands of "brilliance". Researchers should be strongly "incentivized" to make their data and source code available and immediately usable (ie, not in Matlab). And the only way to do that is to gradually devalue data-less, source-code-less research in the way non-peer-review research is now devalued. (and yes, it will be messy, embarrassing, fragile code sometimes but we'll have to live with that). The amazing thing is that small, for-profit, private research organizations can still operate with this "what we produce is a pdf with a picture of our research on it" attitude. Btw, one organization can and I think is help is PLOS: http://www.plos.org/ http://www.plos.org/

robbles 16y ago

sounds like a good application for Fluidinfo: http://www.fluidinfo.com/about/ http://www.fluidinfo.com/about/ if they made it a bit more accessible, scientists could potentially share some of their data, and new research could associate their related findings with the original data.