5 ms·

Here's what makes me sad about this. There are probably useful bits of information collected by social scientists all the time that will never see the light of

Here's what makes me sad about this. There are probably useful bits of information collected by social scientists all the time that will never see the light of day because there is no requirement to publish the data. But more than that, it simply doesn't matter what format the data is in, or if its well presented. It just has to be accessible. As the ability and scope of data mining methods grows, every piece of scientific data will be fodder, not for humans, but for machines looking back over the entirety of scientific output to draw new conclusions from information we simply ignore today. I would go as far as to say that if you feel your work has any value at all, you have a duty to make your data available, so that it can be mined at a later date.

slashcom 13y ago
My understanding of why this isn't often the case in the Social Sciences is that data is usually extremely expensive to collect. The researchers usually spend many, many hours writing proposals, revising it, resubmitting it, until they finally have the funding to collect the data their interested in. Then they actually have to get people into a laboratory, or send someone out to do field work (sometimes halfway across the world), or any number of expensive procedures. And then, after data is collected, they begin analyzing it and seeing what are the interesting patterns in the data. And this takes a lot of time, and often you just want to show one thing at a time. In the current environment of publish-or-perish, and grant providers asking what you've done with their money, you might want to take that data you worked so hard to obtain, and milk quite a few publications out of it. If you openly release the data, then others can easily find interesting patterns in your data before you can, beating you to publication, and diminishing your track record for future grants, tenure, etc. So your point is extremely valid. But the counter point of why open data isn't feasible is, if a researcher does all the hard legwork necessary to get to the interesting analysis stage, why shouldn't they reap the rewards of their hard work (namely, publications and recognition). A few things have been proposed: verification projects (one particular other research group gains access to your data and verifies your results, but is not allowed to use the data otherwise), and grace periods (you have to release your data 3 years after a study or something, giving you time to milk out publications, but still allowing for general verification later). Generally speaking, true scientific verification should also involve the complete recollection of the data. But when data collection is extremely expensive, this is typically infeasible. [I work in a field that's almost all open access, and research data is almost always available upon request. So I agree with you. But data is typically incredibly cheap in my field, so the problem above doesn't really apply.]
- marcosdumay 13y ago
  > A few things have been proposed: verification projects, and grace periods. Is glorifying (and sending grants to) data collectors on the list?
  - slashcom 13y ago
    You mean to say, granting more wide recognition for the collection itself, rather than the analysis stemming from collection? Possibly, I don't know. Mostly this is just my understanding from talking to peers in the social sciences and asking why they don't have open access like my field. I think the problem there though, is knowing what questions to ask (what data to collect) is the hardest part, but fits much more in line with the analysis aspect (ie forming a hypothesis) than collection (conducting a designed experiment). So an ideal researcher would show good work in both areas, not just one.
    - marcosdumay 13y ago
      Exactly that. We can quite well be at awe to people that managed to collect difficult sets of data, instead of the people that made a superficial analysis on it and discovered something. While still reserving some respect for the people that do difficult analysis, of course. That problem is not specific to the social sciences. As far as I know, only physics are making some inroads on that change of attitude, but even there it's restricted to a few fields where data collection is extremely expensive (much more than at social sciences). I think that's one much needed change to adapt science to our current world.
- wycx 13y ago
  This state of affairs is completely arse backwards. In hard sciences, where direct quantitative measurement is made (e.g. the sulfur isotopic composition of a mineral in a rock), those measurements, and details of how they were made are published in the paper. If you have the paper you have the data. How is it that 'sciences' that do not make direct measurements, and instead rely on statistical analysis to separate weak signals from noise in very very indirect measurements can get away _without_ publishing raw data? The burden of proof must be higher for such weak signals, and that must include the raw data (e.g. all the survey results). I am unsurprised people get away with making shit up in these disciplines if we esentially are taking their word for it.

slashcom 13y ago

My understanding of why this isn't often the case in the Social Sciences is that data is usually extremely expensive to collect. The researchers usually spend many, many hours writing proposals, revising it, resubmitting it, until they finally have the funding to collect the data their interested in. Then they actually have to get people into a laboratory, or send someone out to do field work (sometimes halfway across the world), or any number of expensive procedures. And then, after data is collected, they begin analyzing it and seeing what are the interesting patterns in the data. And this takes a lot of time, and often you just want to show one thing at a time. In the current environment of publish-or-perish, and grant providers asking what you've done with their money, you might want to take that data you worked so hard to obtain, and milk quite a few publications out of it. If you openly release the data, then others can easily find interesting patterns in your data before you can, beating you to publication, and diminishing your track record for future grants, tenure, etc. So your point is extremely valid. But the counter point of why open data isn't feasible is, if a researcher does all the hard legwork necessary to get to the interesting analysis stage, why shouldn't they reap the rewards of their hard work (namely, publications and recognition). A few things have been proposed: verification projects (one particular other research group gains access to your data and verifies your results, but is not allowed to use the data otherwise), and grace periods (you have to release your data 3 years after a study or something, giving you time to milk out publications, but still allowing for general verification later). Generally speaking, true scientific verification should also involve the complete recollection of the data. But when data collection is extremely expensive, this is typically infeasible. [I work in a field that's almost all open access, and research data is almost always available upon request. So I agree with you. But data is typically incredibly cheap in my field, so the problem above doesn't really apply.]

marcosdumay 13y ago
> A few things have been proposed: verification projects, and grace periods. Is glorifying (and sending grants to) data collectors on the list?
- slashcom 13y ago
  You mean to say, granting more wide recognition for the collection itself, rather than the analysis stemming from collection? Possibly, I don't know. Mostly this is just my understanding from talking to peers in the social sciences and asking why they don't have open access like my field. I think the problem there though, is knowing what questions to ask (what data to collect) is the hardest part, but fits much more in line with the analysis aspect (ie forming a hypothesis) than collection (conducting a designed experiment). So an ideal researcher would show good work in both areas, not just one.
  - marcosdumay 13y ago
    Exactly that. We can quite well be at awe to people that managed to collect difficult sets of data, instead of the people that made a superficial analysis on it and discovered something. While still reserving some respect for the people that do difficult analysis, of course. That problem is not specific to the social sciences. As far as I know, only physics are making some inroads on that change of attitude, but even there it's restricted to a few fields where data collection is extremely expensive (much more than at social sciences). I think that's one much needed change to adapt science to our current world.
wycx 13y ago
This state of affairs is completely arse backwards. In hard sciences, where direct quantitative measurement is made (e.g. the sulfur isotopic composition of a mineral in a rock), those measurements, and details of how they were made are published in the paper. If you have the paper you have the data. How is it that 'sciences' that do not make direct measurements, and instead rely on statistical analysis to separate weak signals from noise in very very indirect measurements can get away _without_ publishing raw data? The burden of proof must be higher for such weak signals, and that must include the raw data (e.g. all the survey results). I am unsurprised people get away with making shit up in these disciplines if we esentially are taking their word for it.