4 ms·

I think the title they chose is just misleading (wrong?). Otherwise, their conclusion is > we found that articles downloaded from Sci-hub were cited 1.72 times

I think the title they chose is just misleading (wrong?). Otherwise, their conclusion is > we found that articles downloaded from Sci-hub were cited 1.72 times more than papers not downloaded from Sci-hub and that the number of downloads from Sci-hub was a robust predictor of future citations > The results suggest that limited access to publications may limit some scientific research from achieving its full impact.

oefrha 6y ago
It’s more than just the title. The conclusion from abstract reads “The results suggest that limited access to publications may limit some scientific research from achieving its full impact.” My uncharitable read is they slapped an interesting yet unsupported conclusion onto a rather boring observation. Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim.
- jaekash 6y ago
  > Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim. I was not sure about this, but if this is the case then their conclusion seems to be unrelated to their findings.
  - oefrha 6y ago
    I'm pretty sure about this now. I just looked into the actual data set[1], and analyzed a few papers from it, e.g. [2] the first paper in scopus_nature.csv. They're definitely available on Sci-Hub at least at the moment [3], and there's no reason to believe they weren't available back then. Also, here's what they say about the dataset, emphasis mine: > As a quality control, we performed a random sampling of all the articles retrieved, excluding those already present in the first data set. As in this second data set, the number of Sci-hub downloads is precisely equal to zero, we regard it as a control group (nC = 4,015) from which we are going to estimate comparisons for our experimental group (nE = 4,646). Downloads equal to zero != not available. [1] https://osf.io/xb9yn/ https://osf.io/xb9yn/ [2] https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141654&partnerID=40&md5=387eaeb70ac807a4fa4acb573e42b03a https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141... [3] https://sci-hub.tw/10.1038/541123a https://sci-hub.tw/10.1038/541123a
    - amatic 6y ago
      > Downloads equal to zero != not available. I'm wondering about this too. What could be the possible other reason for the papers not being downloaded? Boring titles? Preprints easily available? Open access? > and there's no reason to believe they weren't available back then. Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub?
      - oefrha 6y ago
        > Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub? That's within the realm of possibilities but sure doesn't sound like the case the way they described their methodology. That would have required a catalog of available papers on Sci-Hub too, but the dataset they used only contained download logs, so they couldn't have known whether a zero-download paper was available or not (unless I misunderstood what the first dataset is about).
    - Beldin 6y ago
      So it seems that this papers result comes to "more downloads => more citations". Of course, that is hardly a surprising result. I think most authors have read most papers they cite (although things like reviewer/editor shenanigans could make this less than 100%). Of course, such a pedestrian result does not grab attention.
barbegal 6y ago
I thought that too. But in the "Statistical analyses" section it says: "the interpretation of our robust estimates goes beyond claiming mere correlations or associations." which when you read through the statistical analysis is clearly not true.
mistaken 6y ago
I think what they're hinting at is that without SciHub, even a popular paper would have less exposure. Some papers which cite a popular publication can only do so, because they could download the referenced one through SciHub.
- subroutine 6y ago
  I don't think anyone questions that some articles have been cited because they were downloaded from scihub. This was still a garbage attempt to 'prove' this conclusion. And I don't have any faith the authors measured the true magnitude of these effects.

oefrha 6y ago

It’s more than just the title. The conclusion from abstract reads “The results suggest that limited access to publications may limit some scientific research from achieving its full impact.” My uncharitable read is they slapped an interesting yet unsupported conclusion onto a rather boring observation. Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim.

jaekash 6y ago
> Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim. I was not sure about this, but if this is the case then their conclusion seems to be unrelated to their findings.
- oefrha 6y ago
  I'm pretty sure about this now. I just looked into the actual data set[1], and analyzed a few papers from it, e.g. [2] the first paper in scopus_nature.csv. They're definitely available on Sci-Hub at least at the moment [3], and there's no reason to believe they weren't available back then. Also, here's what they say about the dataset, emphasis mine: > As a quality control, we performed a random sampling of all the articles retrieved, excluding those already present in the first data set. As in this second data set, the number of Sci-hub downloads is precisely equal to zero, we regard it as a control group (nC = 4,015) from which we are going to estimate comparisons for our experimental group (nE = 4,646). Downloads equal to zero != not available. [1] https://osf.io/xb9yn/ https://osf.io/xb9yn/ [2] https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141654&partnerID=40&md5=387eaeb70ac807a4fa4acb573e42b03a https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141... [3] https://sci-hub.tw/10.1038/541123a https://sci-hub.tw/10.1038/541123a
  - amatic 6y ago
    > Downloads equal to zero != not available. I'm wondering about this too. What could be the possible other reason for the papers not being downloaded? Boring titles? Preprints easily available? Open access? > and there's no reason to believe they weren't available back then. Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub?
    - oefrha 6y ago
      > Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub? That's within the realm of possibilities but sure doesn't sound like the case the way they described their methodology. That would have required a catalog of available papers on Sci-Hub too, but the dataset they used only contained download logs, so they couldn't have known whether a zero-download paper was available or not (unless I misunderstood what the first dataset is about).
  - Beldin 6y ago
    So it seems that this papers result comes to "more downloads => more citations". Of course, that is hardly a surprising result. I think most authors have read most papers they cite (although things like reviewer/editor shenanigans could make this less than 100%). Of course, such a pedestrian result does not grab attention.

barbegal 6y ago

I thought that too. But in the "Statistical analyses" section it says: "the interpretation of our robust estimates goes beyond claiming mere correlations or associations." which when you read through the statistical analysis is clearly not true.

mistaken 6y ago

I think what they're hinting at is that without SciHub, even a popular paper would have less exposure. Some papers which cite a popular publication can only do so, because they could download the referenced one through SciHub.

subroutine 6y ago
I don't think anyone questions that some articles have been cited because they were downloaded from scihub. This was still a garbage attempt to 'prove' this conclusion. And I don't have any faith the authors measured the true magnitude of these effects.