4 ms·

For an idea of what some of the backlash about the NEJM editorial looks like, simply check Twitter for #researchparasites: https://twitter.com/search?q=%23resea

by pak 11y ago

For an idea of what some of the backlash about the NEJM editorial looks like, simply check Twitter for #researchparasites: https://twitter.com/search?q=%23researchparasites https://twitter.com/search?q=%23researchparasites -- and for a quicker version, some of the most influential voices are summarized here: http://www.forbes.com/sites/davidshaywitz/2016/01/21/data-scientists-research-parasites http://www.forbes.com/sites/davidshaywitz/2016/01/21/data-sc... The paragraph from NEJM's piece that drew the brunt of the scorn was the following: A second concern held by some is that a new class of research person will emerge — people who had nothing to do with the design and execution of the study but use another group’s data for their own ends, possibly stealing from the research productivity planned by the data gatherers, or even use the data to try to disprove what the original investigators had posited. There is concern among some front-line researchers that the system will be taken over by what some researchers have characterized as “research parasites.” The meme has already given birth to at least one parody account: https://twitter.com/dataparasite https://twitter.com/dataparasite and one rather tongue-in-cheek domain purchase: http://researchparasites.com http://researchparasites.com

Satifer 11y ago
"...or even use the data to try to disprove what the original investigators had posited." How is this a bad thing? The original investigators - as scientists and seekers of the valid, the factual, the accurate - should welcome competing attempts, especially given that such attempts may reveal a faulty original analysis.
- Satifer 11y ago
  ... there is just so much wrong with that editorial
- c3534l 11y ago
  Indeed. It is such a bizarrely anti-science attitude. I suspect the mindset is not about advancing scientific discourse, but maintaining the sense of prestige the journal's name carries.
  - michaelhoffman 11y ago
    I'm not sure how well that prestige is earned. NEJM has to retract substantially more papers than any other high-profile journal: http://retractionwatch.com/2011/08/11/is-it-time-for-a-retraction-index/ http://retractionwatch.com/2011/08/11/is-it-time-for-a-retra... If the papers they publish get more scrutiny, they will undoubtedly have to retract even more.
- IanCal 11y ago
  I wonder if this is not a concern with high quality rebuttals but with poor, fast analysis conducted with an agenda. The worry that some people will root through the data to find any small issues and blow them far out of proportion. I'm not sure I agree with it, but I think it's possible that's more what they're talking about. Rather than, say, a lab coming back and saying "we analysed the BICEP data carefully and found the results are possibly the result of dust".
Outdoorsman 11y ago
Though not a scientist or researcher, by any measure, I like to think that I understand the viewpoint of both positions...it seems there are two that are quite distinct: 1) Defenders of one of the tenets of the Scientific Method--peer review--and the transparency that that concept demands... 2)Those wary of what the current environment implies might befall those who are totally transparent with both research data and results... "Research Parisites" are being deemed as synonymous with "Patent Trolls"... It will be interesting to see how this is sorted...competition is fierce for funding...important fundamental principles, and billions of dollars, are stake...
- nonbel 11y ago
  Peer review is a tenet of the Scientific Method? I don't think so. Independent replication definitely has been considered so: https://en.wikipedia.org/wiki/Nullius_in_verba https://en.wikipedia.org/wiki/Nullius_in_verba I guess that could be considered a specific type of peer review. Getting useful feedback is always good too, whether related to science or not. Maybe of interest: http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review/ http://michaelnielsen.org/blog/three-myths-about-scientific-...
chris_wot 11y ago
Oh brother. They are literally putting science and medicine back by decades. All those conspiracy theorists who won't vaccinate their children because of "big pharmacy" hiding data now can use this editorial, with some credibility, to say that scientific studies are biased and lack transparency. The authors of this editorial should be stood down.
kriro 11y ago
I think the entire concept of "stealing from the research productivity..." is absurd. Repeated analysis of my data set only improves the quality of my work by strenthening the argument via a fresh set of eyeballs or showing flaws in my work which I can imrpove upon...both of which are things I want as a scientist. I feel like hidden datasets are a real problem. In fields like medicine where some authority will look at the data eventually there's at least some quality control (debatable as to how much) but there's entire sections of science were it would be rather trivial to fake entire studies without even collecting data...especially since redoing experiments seems to have a bad rep, too. Pretty broken system overall. I'd love if there was a step in the acceptance of papers that would say...paper accepted under the provision that the data set is made available. You still get "the glory" because the stuff is published and you're the first source on it but now the data is also available. Additionally, one of the goals of science is being reproducible and transparent. If an experiment is well described and reproducible, additive data sets could be built. Run a similar enough experiment but with another demographic, add to the data set...etc. Edit: I also don't see why the role of "data gatherer" can't be more prestigious. I mean sure, traditionally you gather data for a reason and want to answer a research question with it and that's what you're judged by. However there's tremendous value in identifying that no good data set exists for some area and then outlining a solid, transparent and scientific process of collecting that data and executing it. I'd call that a valuable paper even if no hypotheses are tested. As long as there's a place where you can publish those papers and make the data available it would probably also be a paper that is good for your career by a metric that seems to matter a lot. You'd potentially get a lot of citations since everyone who conducts analysis based on the data set or extends it etc. would cite you.
- IanCal 11y ago
  > Edit: I also don't see why the role of "data gatherer" can't be more prestigious. I agree wholeheartedly. If datasets are so valuable that lots of people can easily capitalise on them and produce great science, then the creation of those datasets should be rewarded similarly to an extremely valuable paper. Machine learning seems to be a field that this is going pretty well in, people are publishing their models and datasets more so that I can grab a trained model of a huge image recognition neural net and try it out on my own data. > As long as there's a place where you can publish those papers and make the data available it would probably also be a paper that is good for your career by a metric that seems to matter a lot. I'd personally like to see a shift away from requiring a paper to cite a dataset. But I don't think that really alters your point. (disclaimer, I work for Digital Science which is a parent company of figshare, but this is a personal opinion)

by pak 11y ago

Satifer 11y ago

"...or even use the data to try to disprove what the original investigators had posited." How is this a bad thing? The original investigators - as scientists and seekers of the valid, the factual, the accurate - should welcome competing attempts, especially given that such attempts may reveal a faulty original analysis.

Satifer 11y ago
... there is just so much wrong with that editorial
c3534l 11y ago
Indeed. It is such a bizarrely anti-science attitude. I suspect the mindset is not about advancing scientific discourse, but maintaining the sense of prestige the journal's name carries.
- michaelhoffman 11y ago
  I'm not sure how well that prestige is earned. NEJM has to retract substantially more papers than any other high-profile journal: http://retractionwatch.com/2011/08/11/is-it-time-for-a-retraction-index/ http://retractionwatch.com/2011/08/11/is-it-time-for-a-retra... If the papers they publish get more scrutiny, they will undoubtedly have to retract even more.
IanCal 11y ago
I wonder if this is not a concern with high quality rebuttals but with poor, fast analysis conducted with an agenda. The worry that some people will root through the data to find any small issues and blow them far out of proportion. I'm not sure I agree with it, but I think it's possible that's more what they're talking about. Rather than, say, a lab coming back and saying "we analysed the BICEP data carefully and found the results are possibly the result of dust".

Outdoorsman 11y ago

Though not a scientist or researcher, by any measure, I like to think that I understand the viewpoint of both positions...it seems there are two that are quite distinct: 1) Defenders of one of the tenets of the Scientific Method--peer review--and the transparency that that concept demands... 2)Those wary of what the current environment implies might befall those who are totally transparent with both research data and results... "Research Parisites" are being deemed as synonymous with "Patent Trolls"... It will be interesting to see how this is sorted...competition is fierce for funding...important fundamental principles, and billions of dollars, are stake...

nonbel 11y ago
Peer review is a tenet of the Scientific Method? I don't think so. Independent replication definitely has been considered so: https://en.wikipedia.org/wiki/Nullius_in_verba https://en.wikipedia.org/wiki/Nullius_in_verba I guess that could be considered a specific type of peer review. Getting useful feedback is always good too, whether related to science or not. Maybe of interest: http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review/ http://michaelnielsen.org/blog/three-myths-about-scientific-...

chris_wot 11y ago

Oh brother. They are literally putting science and medicine back by decades. All those conspiracy theorists who won't vaccinate their children because of "big pharmacy" hiding data now can use this editorial, with some credibility, to say that scientific studies are biased and lack transparency. The authors of this editorial should be stood down.

kriro 11y ago

I think the entire concept of "stealing from the research productivity..." is absurd. Repeated analysis of my data set only improves the quality of my work by strenthening the argument via a fresh set of eyeballs or showing flaws in my work which I can imrpove upon...both of which are things I want as a scientist. I feel like hidden datasets are a real problem. In fields like medicine where some authority will look at the data eventually there's at least some quality control (debatable as to how much) but there's entire sections of science were it would be rather trivial to fake entire studies without even collecting data...especially since redoing experiments seems to have a bad rep, too. Pretty broken system overall. I'd love if there was a step in the acceptance of papers that would say...paper accepted under the provision that the data set is made available. You still get "the glory" because the stuff is published and you're the first source on it but now the data is also available. Additionally, one of the goals of science is being reproducible and transparent. If an experiment is well described and reproducible, additive data sets could be built. Run a similar enough experiment but with another demographic, add to the data set...etc. Edit: I also don't see why the role of "data gatherer" can't be more prestigious. I mean sure, traditionally you gather data for a reason and want to answer a research question with it and that's what you're judged by. However there's tremendous value in identifying that no good data set exists for some area and then outlining a solid, transparent and scientific process of collecting that data and executing it. I'd call that a valuable paper even if no hypotheses are tested. As long as there's a place where you can publish those papers and make the data available it would probably also be a paper that is good for your career by a metric that seems to matter a lot. You'd potentially get a lot of citations since everyone who conducts analysis based on the data set or extends it etc. would cite you.

IanCal 11y ago
> Edit: I also don't see why the role of "data gatherer" can't be more prestigious. I agree wholeheartedly. If datasets are so valuable that lots of people can easily capitalise on them and produce great science, then the creation of those datasets should be rewarded similarly to an extremely valuable paper. Machine learning seems to be a field that this is going pretty well in, people are publishing their models and datasets more so that I can grab a trained model of a huge image recognition neural net and try it out on my own data. > As long as there's a place where you can publish those papers and make the data available it would probably also be a paper that is good for your career by a metric that seems to matter a lot. I'd personally like to see a shift away from requiring a paper to cite a dataset. But I don't think that really alters your point. (disclaimer, I work for Digital Science which is a parent company of figshare, but this is a personal opinion)