8 ms·

The system within which many researchers must work to advance their careers is fundamentally at odds with sharing data. Collecting data is unglamorous, laboriou

by rtl49 11y ago

The system within which many researchers must work to advance their careers is fundamentally at odds with sharing data. Collecting data is unglamorous, laborious, and often very expensive. This places much data collection firmly within the purview of research institutions, where resources and cheap labor are abundant. To progress in this context, one must publish. It isn't enough only to have collected the data; the researcher must also be the one to analyze it, and to do so before someone else does. Thus for any self-interested person in research, there is no incentive to disclose hard-won data to the public at large, effectively crowd-sourcing analysis and foregoing the career advantages and opportunities that a large store of useful data offers. One may find it unethical, but few who have committed many years of hard labor to advance in a research context would do otherwise.

taurath 11y ago
Very sad to hear. Too bad so much focus is on the individual - advancing ones career is at odds with advancing the science that you care about.
- grandinj 11y ago
  It's not just career advancement, it's often also career survival. No papers - no funding. No funding - no job.
  - taurath 11y ago
    I wonder how much the bottleneck of resources (money/etc) would be trimmed down if centers of research were spread throughout the country rather than in very expensive real estate IE on a college campus? I'm probably super super over simplifying the problem.
hannob 11y ago
So what you're saying is essentially that the point of science is to advance careers, not to find insights and come closer to the truth? It may very well be true, but that would be a devastating account of the situation of today's science. And it would desperately need to be fixed.
- seanmcdirmid 11y ago
  There are plenty in it just for the career, gaming the incentives of an imperfect system. But not all are like this, it is usually obvious (at least in my field) who is real and who is just collecting a check.
- sdgsfsfgfg 11y ago
  This is a devastating account of today's society.
- stevetrewick 11y ago
  I don't think the parent is saying that's what science is for, but rather that this is what the incentive structure currently looks like. We mythologise science and scientists as being above this, but they aren't. Incentives matter.
- ProblemFactory 11y ago
  No, and all scientists want everyone to find insights and come closer to truth. But the reality is that both the career advancement and funding for further experiments is almost entirely based on the number of publications in prestigious journals. And effort put into collecting data is not by itself publishable: you must add analysis, and very often analysis that successfully confirms a new hypothesis or invalidates previous beliefs to publish. Journals prefer to publish exciting results, and "we ran experiment X, and found no significant effects" is usually rejected. Fixing this is desperately needed, but so far nobody has come up with a solution that would actually fix this. A middle ground would be to require data publication after a certain time period. For example, NASA and ESA release original data 6-12 months after collection. Releasing original data together with the paper also works.
  - ata_aman 11y ago
    Exactly. I think publishing the data _along_ your publication is the best way to go about it.
  - eru 11y ago
    > Fixing this is desperately needed, but so far nobody has come up with a solution that would actually fix this. Wouldn't rewarding people for `cites' of their dataset be a way to fix this? Ie if someone uses your data, they have to cite you, and that's just as good as someone citing an analysis paper.
YeGoblynQueenne 11y ago
Edit: ah, never mind- bit emotional this morning. Covered by hannob below.
pcrh 11y ago
It isn't such a terrible thing. Unlike corporations, which routinely withhold data generated in clinical trials, academic researchers must eventually publish their data, otherwise their careers will suffer.
- stevetrewick 11y ago
  AIUI these researchers are complaining about what might become of their data after they have published their studies. Edit: no, on closer reading, I am incorrect. My bad.
- RA_Fisher 11y ago
  I follow JAMA's publications and routinely ask / seek data from authors (politely). I've never once been sent or guided towards data. I asked Dr. Vivian Fonseca for the data associated with this study: http://www.ncbi.nlm.nih.gov/pubmed/23218892 http://www.ncbi.nlm.nih.gov/pubmed/23218892 The text is free and you'll see he says on page 1, "the raw data are available from Pamlab LLC." I asked Pamlab for the data and they stonewalled. I asked Dr. Fonseca and he stonewalled. I asked the Journal and they said, "Not our job." And that's an article that says the data are available!
  - kefka 11y ago
    That kind of a situation seems pretty simple: request data upon review, and actively deny the paper until data has been submitted. Or shame them academically. You'd be surprised how far that goes.
  - michaelhoffman 11y ago
    This is why it is so important to ensure data is deposited in an independent repository at time of publication. It is too easy to claim the data will be made available and then simply not do it, and there is little recourse. It would be really worthwhile for the International Committee of Medical Journal Editors to hear your story, as they are proposing that authors should be allowed to withhold data for six months after publication. See my comment elsewhere in this thread: https://news.ycombinator.com/item?id=11019760 https://news.ycombinator.com/item?id=11019760 If you have access to PubMed Commons, I would post a comment warning others that the authors refused to make the data available despite promising to do so in the paper, and that people should treat the results with caution.
teekert 11y ago
Having a lot of data is a big incentive to find someone who can analyze it correctly because it will raise the impact of your scientific output. It is not as bad as people describe here. Even large companies need to publish and share in order to show that they know their business and in order to have any shot at a collaboration wit a large hospital (or another company). Such a hospital will also want to publish and share in general. A publication of a company also signals: "Hey come to us, maybe we can work out a deal and provide a nice service based on our work". Yes, there are NDAs, contracts and patents but that company paid a lot of people for their efforts with only a small chance at ROI, it is only fair they are rewarded. An optimum is often found that favors sharing as little as possible but often "as little as possible" is still quite a lot. Also, how can people label this as unethical when people would not even have gathered the data if they knew beforehand someone would force them into sharing, diminishing the data's return on investment? I'm also pretty sure the sharing of a complete data set will get you a lot of citations, so there is a financial incentive next to the moral one.
rlvesco7 11y ago
One way to solve this problem is to create a data license that allows data sharing for replication use only until some embargo date set by the researcher. This would allow external researchers to immediately replicate published results, but would prohibit them from unfairly benefiting from someone else's hard work.
- chrisamiller 11y ago
  This exists and has been used successfully in a number of large consortia, like the Cancer Genome Atlas. It essentially says that the data goes live immediately, but the researchers who generated the data have exclusive publishing rights for 6 months from that date. After that, it's fair game for anyone.
michaelhoffman 11y ago
It's fine for researchers, in most cases, to keep the data to themselves until publication. This allows them to do this analysis first. But many clinical trial researchers want to withhold the data even after publication. This is fundamentally at odds with how science works. It means no one can independently evaluate the results.

by rtl49 11y ago

taurath 11y ago

Very sad to hear. Too bad so much focus is on the individual - advancing ones career is at odds with advancing the science that you care about.

grandinj 11y ago
It's not just career advancement, it's often also career survival. No papers - no funding. No funding - no job.
- taurath 11y ago
  I wonder how much the bottleneck of resources (money/etc) would be trimmed down if centers of research were spread throughout the country rather than in very expensive real estate IE on a college campus? I'm probably super super over simplifying the problem.

hannob 11y ago

So what you're saying is essentially that the point of science is to advance careers, not to find insights and come closer to the truth? It may very well be true, but that would be a devastating account of the situation of today's science. And it would desperately need to be fixed.

seanmcdirmid 11y ago
There are plenty in it just for the career, gaming the incentives of an imperfect system. But not all are like this, it is usually obvious (at least in my field) who is real and who is just collecting a check.
sdgsfsfgfg 11y ago
This is a devastating account of today's society.
stevetrewick 11y ago
I don't think the parent is saying that's what science is for, but rather that this is what the incentive structure currently looks like. We mythologise science and scientists as being above this, but they aren't. Incentives matter.
ProblemFactory 11y ago
No, and all scientists want everyone to find insights and come closer to truth. But the reality is that both the career advancement and funding for further experiments is almost entirely based on the number of publications in prestigious journals. And effort put into collecting data is not by itself publishable: you must add analysis, and very often analysis that successfully confirms a new hypothesis or invalidates previous beliefs to publish. Journals prefer to publish exciting results, and "we ran experiment X, and found no significant effects" is usually rejected. Fixing this is desperately needed, but so far nobody has come up with a solution that would actually fix this. A middle ground would be to require data publication after a certain time period. For example, NASA and ESA release original data 6-12 months after collection. Releasing original data together with the paper also works.
- ata_aman 11y ago
  Exactly. I think publishing the data _along_ your publication is the best way to go about it.
- eru 11y ago
  > Fixing this is desperately needed, but so far nobody has come up with a solution that would actually fix this. Wouldn't rewarding people for `cites' of their dataset be a way to fix this? Ie if someone uses your data, they have to cite you, and that's just as good as someone citing an analysis paper.

YeGoblynQueenne 11y ago

Edit: ah, never mind- bit emotional this morning. Covered by hannob below.

pcrh 11y ago

It isn't such a terrible thing. Unlike corporations, which routinely withhold data generated in clinical trials, academic researchers must eventually publish their data, otherwise their careers will suffer.

stevetrewick 11y ago
AIUI these researchers are complaining about what might become of their data after they have published their studies. Edit: no, on closer reading, I am incorrect. My bad.
RA_Fisher 11y ago
I follow JAMA's publications and routinely ask / seek data from authors (politely). I've never once been sent or guided towards data. I asked Dr. Vivian Fonseca for the data associated with this study: http://www.ncbi.nlm.nih.gov/pubmed/23218892 http://www.ncbi.nlm.nih.gov/pubmed/23218892 The text is free and you'll see he says on page 1, "the raw data are available from Pamlab LLC." I asked Pamlab for the data and they stonewalled. I asked Dr. Fonseca and he stonewalled. I asked the Journal and they said, "Not our job." And that's an article that says the data are available!
- kefka 11y ago
  That kind of a situation seems pretty simple: request data upon review, and actively deny the paper until data has been submitted. Or shame them academically. You'd be surprised how far that goes.
- michaelhoffman 11y ago
  This is why it is so important to ensure data is deposited in an independent repository at time of publication. It is too easy to claim the data will be made available and then simply not do it, and there is little recourse. It would be really worthwhile for the International Committee of Medical Journal Editors to hear your story, as they are proposing that authors should be allowed to withhold data for six months after publication. See my comment elsewhere in this thread: https://news.ycombinator.com/item?id=11019760 https://news.ycombinator.com/item?id=11019760 If you have access to PubMed Commons, I would post a comment warning others that the authors refused to make the data available despite promising to do so in the paper, and that people should treat the results with caution.

teekert 11y ago

Having a lot of data is a big incentive to find someone who can analyze it correctly because it will raise the impact of your scientific output. It is not as bad as people describe here. Even large companies need to publish and share in order to show that they know their business and in order to have any shot at a collaboration wit a large hospital (or another company). Such a hospital will also want to publish and share in general. A publication of a company also signals: "Hey come to us, maybe we can work out a deal and provide a nice service based on our work". Yes, there are NDAs, contracts and patents but that company paid a lot of people for their efforts with only a small chance at ROI, it is only fair they are rewarded. An optimum is often found that favors sharing as little as possible but often "as little as possible" is still quite a lot. Also, how can people label this as unethical when people would not even have gathered the data if they knew beforehand someone would force them into sharing, diminishing the data's return on investment? I'm also pretty sure the sharing of a complete data set will get you a lot of citations, so there is a financial incentive next to the moral one.

rlvesco7 11y ago

One way to solve this problem is to create a data license that allows data sharing for replication use only until some embargo date set by the researcher. This would allow external researchers to immediately replicate published results, but would prohibit them from unfairly benefiting from someone else's hard work.

chrisamiller 11y ago
This exists and has been used successfully in a number of large consortia, like the Cancer Genome Atlas. It essentially says that the data goes live immediately, but the researchers who generated the data have exclusive publishing rights for 6 months from that date. After that, it's fair game for anyone.

michaelhoffman 11y ago

It's fine for researchers, in most cases, to keep the data to themselves until publication. This allows them to do this analysis first. But many clinical trial researchers want to withhold the data even after publication. This is fundamentally at odds with how science works. It means no one can independently evaluate the results.