13 ms·

The Sci-Hub Effect: Sci-Hub downloads lead to more article citations

754 points by lnyan 6y ago

enjoyyourlife 6y ago
Appropriately posted on arXiv
oefrha 6y ago
Maybe I misunderstood something, but it seems to me their data demonstrate that less interesting papers that nobody bothered to download from SciHub tend to have fewer citations than more interesting ones that are downloaded? How does that support the conclusion “Sci-Hub downloads lead to more article citations” (emphasis mine)? (Yes I read the paper and I’ve seen all the other factors they considered, but those factors simply aren’t as good predictors/indicators of interest as an organic download number from any channel, whether it’s ScienceDirect or SciHub.) Edit: To be clear, papers in their supposedly "limited access" (if we want to draw the same conclusion) control group appear to be available on Sci-Hub, i.e. as open access as their Sci-Hub group. Details in this comment: https://news.ycombinator.com/item?id=23710992 https://news.ycombinator.com/item?id=23710992
- Noluris 6y ago
  I think the title they chose is just misleading (wrong?). Otherwise, their conclusion is > we found that articles downloaded from Sci-hub were cited 1.72 times more than papers not downloaded from Sci-hub and that the number of downloads from Sci-hub was a robust predictor of future citations > The results suggest that limited access to publications may limit some scientific research from achieving its full impact.
  - oefrha 6y ago
    It’s more than just the title. The conclusion from abstract reads “The results suggest that limited access to publications may limit some scientific research from achieving its full impact.” My uncharitable read is they slapped an interesting yet unsupported conclusion onto a rather boring observation. Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim.
    - jaekash 6y ago
      > Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim. I was not sure about this, but if this is the case then their conclusion seems to be unrelated to their findings.
      - oefrha 6y ago
        I'm pretty sure about this now. I just looked into the actual data set[1], and analyzed a few papers from it, e.g. [2] the first paper in scopus_nature.csv. They're definitely available on Sci-Hub at least at the moment [3], and there's no reason to believe they weren't available back then. Also, here's what they say about the dataset, emphasis mine: > As a quality control, we performed a random sampling of all the articles retrieved, excluding those already present in the first data set. As in this second data set, the number of Sci-hub downloads is precisely equal to zero, we regard it as a control group (nC = 4,015) from which we are going to estimate comparisons for our experimental group (nE = 4,646). Downloads equal to zero != not available. [1] https://osf.io/xb9yn/ https://osf.io/xb9yn/ [2] https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141654&partnerID=40&md5=387eaeb70ac807a4fa4acb573e42b03a https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141... [3] https://sci-hub.tw/10.1038/541123a https://sci-hub.tw/10.1038/541123a
        amatic 6y ago
        > Downloads equal to zero != not available. I'm wondering about this too. What could be the possible other reason for the papers not being downloaded? Boring titles? Preprints easily available? Open access? > and there's no reason to believe they weren't available back then. Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub?
        oefrha 6y ago
        > Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub? That's within the realm of possibilities but sure doesn't sound like the case the way they described their methodology. That would have required a catalog of available papers on Sci-Hub too, but the dataset they used only contained download logs, so they couldn't have known whether a zero-download paper was available or not (unless I misunderstood what the first dataset is about).
  - barbegal 6y ago
    I thought that too. But in the "Statistical analyses" section it says: "the interpretation of our robust estimates goes beyond claiming mere correlations or associations." which when you read through the statistical analysis is clearly not true.
  - mistaken 6y ago
    I think what they're hinting at is that without SciHub, even a popular paper would have less exposure. Some papers which cite a popular publication can only do so, because they could download the referenced one through SciHub.
    - subroutine 6y ago
      I don't think anyone questions that some articles have been cited because they were downloaded from scihub. This was still a garbage attempt to 'prove' this conclusion. And I don't have any faith the authors measured the true magnitude of these effects.
- deleted 6y ago
  [deleted]
- subroutine 6y ago
  Yes this is a huge confound. Their methods state... The first data set contained all articles (nE = 4,646) that were downloaded from Sci-hub between September 2015 and February 2016 [24, 25]. The second data set was extracted from the Scopus database and contained all articles published in the selected journals within the same time period. From this data set, we excluded articles already present in the first data set. Their methods lend to circular reasoning since the conclusion could also be "papers of high interest (cited more often) are more likely to be downloaded from scihub"
- chiefalchemist 6y ago
  Same here. Simply put, this is correlation. The irony humorous.
- Chris2048 6y ago
  So basically, correlation, speculated (w/o basis) as causation?
- pen2l 6y ago
  As someone from a research lab from a big name school that already has access to every journal: we use sci-hub all the time just because it is so damn convenient (me, I have a chrome addon wherein I click a button on a page that has a doi link, and wahlah, I'm looking at the pdf. Firewall at work blocks scihub, but I find ways around it). As to why my lab might be preferring papers from big-name journal: I think 1) there's admittedly our own bias of wanting to cite big papers vs. small papers, 2) if we're doing some basic literature search, Google will also yield results that are already more popular. Really, at the moment, I struggle to see how our behavior changes about which papers we're citing with or without scihub.
  - neutronicus 6y ago
    I think without Sci-hub I might have just said "fuck it" and never read certain papers from smaller journals that I later wound up citing One thing I would do a lot with Google Scholar is find a highly-cited, somewhat recent (past decade or so) paper and then click into its citations to find the very most recent related work. IIRC this interface was chronological so there was no big-journal bias and I found a lot of stuff in random journals. Low-friction quality-filtering through SciHub was very helpful for me in this process. I worked in a theoretical field, so it's actually possible to assess the quality of a paper just by reading it. If you're doing something experimental you may have to lean on the big-journal filter a little bit more.
  - sitkack 6y ago
    > I struggle to see how our behavior changes about which papers we're citing with or without scihub. Under what conditions do you see that changing? You yourself say how useful it is, but discount the effect it has on the citation graph. Could we say the samething about arxiv? This is exactly why research needs to be open and freely available. The esoteric, rarely cited papers are locked behind paywalls and away from search engines. And in doing so, their impact fades over time as the flock centers around a single vein of focus. The future should look like a net not a river. I am not an academic and I read papers for their metalessons in areas that interest me. But without Sci-Hub, lots of research would not be accessible outside of its niche, esp if it was unlucky enough to not be in the right journal at the right time. On hilarious pattern I see, is when a bigshot in a field publishes a new paper, there is a race to cite it by some nobody. Basically the academic publishing version of "first post".
- dekhn 6y ago
  I find in nearly all cases, if you invert the causality and the results make more sense, you can ignore a paper. "Papers with more citations are downloaded more frequently on Sci-Hub" makes more sense to me than "Papers downloaded frequently from Sci-Hub get more citations".
- credit_guy 6y ago
  I guess what you are saying is that correlation does not imply causation, and in this case the causality arrow may go the other way round. In this case, there's something that actually can help establish the direction of the causality arrow: time. If the citations precede the downloads, then sure, it's more well known papers get downloaded more. This papers show that the downloads precede citations though: "number of downloads from Sci-hub was a robust predictor of future citations" While it's not a watertight argument that downloads lead to citations, the theory is at least plausible.
skybrian 6y ago
It seems like downloads from Sci-Hub and citations could have a common cause, for example, being interesting to scientists. If Sci-Hub downloads are a good proxy for interest and they happen first, then they could usefully be used to predict citations, without necessarily being a cause. Different people could be using SciHub versus making citations. I couldn't tell from the paper whether they considered this, or what they think the cause graph looks like.
- kgwgk 6y ago
  "After controlling for problems of endogeneity, heteroscedasticity, and inconsistency, the interpretation of our robust estimates goes beyond claiming mere correlations or associations." They think they have showed causality. If it only was so easy...
  - barbegal 6y ago
    Yeah it doesn't look like they consulted a statistician when doing this research. It always amazes me how a paper like this can be 90% statistics based but non of the authors are statisticians. Without the proper statistical analysis it just becomes a paper on how highly cited papers get downloaded from Sci-Hub which isn't very interesting.
  - hatsunearu 6y ago
    Whoever reviews this is probably going to go ballistic lol
    - js8 6y ago
      The reviewers should still hedge their bets and make sure their research is available on Sci-Hub.
- fxtentacle 6y ago
  There is a very obvious cause, which is that people tend not to cite papers that they have never seen. So if your reviewer tells you that you need more citations, you use the stuff that's easiest to get.
newyankee 6y ago
I mean nowadays i see a lot of motivated and young crowd in India who have become very knowledgeable due to the explosion in many scientific and other such youtube channels in India. Previously amateur researchers could not get access to research papers due to the prohibitive cost of access (unless they are in a really top notch university, which frankly told is highly limited in India as it is purely a numbers game). Sci-hub really enables access to many such folks. A similar story played out in the last decade (2000 - 2010) when affordable streaming platforms did not exist. Many folks got familiar to global movies and shows due to piracy as the price of even legal DVDs were prohibitively expensive.
- fxtentacle 6y ago
  I think it's also a strong psychological thing. I work for a German company, so paying $40 per paper is no issue. But I still usually close the tab as soon as I see that it's Elsevier. Those people who are interested in me reading their work will send an arxiv link or put the PDF on researchgate or publish open access. Especially for mathematics, I need a printable PDF so that I can take notes. That makes DRMed publications impractical to use. So if someone only links to the paid version of their article, I usually just assume that they're an arrogant prick and skip to the next paper. There's now more good new research being published than I could ever read. Researchers need to adapt to that by reducing friction.
  - huijzer 6y ago
    I would like to promote the Netherlands here. The university I'm working for aims to have all papers open-access. This is derived from national policy [1]. [1] https://en.wikipedia.org/wiki/Open_access_in_the_Netherlands#Policy https://en.wikipedia.org/wiki/Open_access_in_the_Netherlands...
  - suyjuris 6y ago
    You probably know this already, but for the benefit of others: While it depends on the subfield, a lot of computer science and mathematics papers are freely available. 1. Searching for the title often gives you a PDF in the first few results already. Authors usually have the right to upload a version on their personal website or arXiv. 2. There is Google Scholar [1], which has links to PDFs on the right of the result (if it knows one). It is better at this than regular Google search, in my experience. 3. Manually searching on the author's website (or former website, if they moved) sometimes proves successful, although this is relatively more effort. 4. As mentioned, writing an e-mail to the authors works as well. (If you buy a paper, the authors get nothing, so there is no incentive not to share it.) [1] https://scholar.google.com/ https://scholar.google.com/
PudgePacket 6y ago
https://en.wikipedia.org/wiki/FUTON_bias https://en.wikipedia.org/wiki/FUTON_bias > FUTON bias (acronym for "full text on the Net")[1] is a tendency of scholars to cite academic journals with open access—that is, journals that make their full text available on the Internet without charge—in preference to toll-access publications.
TekMol 6y ago
How does one even use Sci-Hub? The search functionality is "temporarily unavailable" and Google seems to have not indexed the site. Find the abstract elsewhere and then use the DOI to find it on Sci-Hub?
- someguyorother 6y ago
  >Find the abstract elsewhere and then use the DOI to find it on Sci-Hub? That's pretty much it. Suppose you want to read a paper on memes, so you search scholar.google.com for "memes". Then let's say you find https://www.jbe-platform.com/content/journals/10.1075/etc.10.1.06lou https://www.jbe-platform.com/content/journals/10.1075/etc.10... (I just chose one at random). Going to the link gives you just the abstract. This one happens to have a full text link, but suppose it didn't. Then you'd go to your favorite sci-hub site (e.g. sci-hub.tw) and paste the URL into the "enter URL..." field, and the paper shows up.
- grive 6y ago
  Yes, pretty much. I find an interesting paper, note its DOI, go to sci-hub to download it. Never tried to search on sci-hub itself.
- nisa 6y ago
  The few times I've needed it due my uni not having access to a paper using the DOI worked - there is also reddit.com/r/scholar
- amatic 6y ago
  For me, this website is usually working, with relatively good search http://gen.lib.rus.ec/scimag/ http://gen.lib.rus.ec/scimag/ Though, I often search on scholar and then c/p the title to scihub. Also, http://z-lib.org http://z-lib.org has a good search function, both for books and articles.
- eythian 6y ago
  I actually quite like using the telegram bot. I just send it a URL or title or whatever and it replies with a PDF.
deleted 6y ago
[deleted]
captn3m0 6y ago
The source for the Sci-Hub downloads is the "Who's downloading pirated papers? Everyone" research from 2016 which was based on anonymized server logs from Sci-Hub. Dataset: https://doi.org/10.5061/dryad.q447c https://doi.org/10.5061/dryad.q447c Research: https://doi.org/10.1126/science.352.6285.508 https://doi.org/10.1126/science.352.6285.508 Article: https://www.sciencemag.org/news/2016/04/whos-downloading-pirated-papers-everyone https://www.sciencemag.org/news/2016/04/whos-downloading-pir...
lowdose 6y ago
Aka the bandwagon effect.
buboard 6y ago
I think the most important is that these citations are being read in full text. This is better that adding a reference based on what it says in the abstract
pedro596 6y ago
It's like saying that if you give free newspapers near the metro it is possible that more people will read them. The only problem is that here they are giving for free "paid newspapers". So, everyone that wanted a paid newspaper but didn't have the money to pay for it read it more times because they were able to steal them. By this analogy the conclusion adds that the fact that people are able to steal newspapers helps to keep everyone more informed. Now, please do the same analogy for food.
- guerrilla 6y ago
  > The only problem is that here they are giving for free "paid newspapers". Or the problem is that there are "paid newspapers" in the first place. > Now, please do the same analogy for food. No, because information is not a finite resources in the same way that food is: Food can't be cloned at negligible cost after it is produced. If food was replicable like in Star Trek, then at that point we could make the same argument for food. (We do waste a lot of food when people can't pay for it or its transporation and distribution though.)
mmis1000 6y ago
Although they do have positive correlation. Not sure whether it is. The more downloads leads to more citations(so the paper is seen by more people). or The more interesting papers has more downloads(people download papers that are more interesting). Looks like both way makes sense. Not sure which way is contributed more to the correlation?
- otherme123 6y ago
  I kind of like the other conclusion: Impact Factor of the journal is not associates with the number of citations. IF is an important point in this bussiness. I've been in some job screening in which they aggregate your papers according to the IF of the journals (like one paper in a 1st quartile journal equals three papers of a 2nd quartile). But according to this data you could publish in a 4th quartile that if your paper is interesting, free to download and with some figures, it will be read and cited.
Yetanfou 6y ago
Ghee, whiz, whod'dathunkit that opening up access to papers would lead to more people having access to papers which they'd then use to write more papers in which they cite those papers they read. Even if you have institutional or individual access to the likes of Wiley or Elsevier it is usually far easier to just feed the DOI to Sci-Hub and read the paper instead of jumping through all the hoops to get 'official' access. This goes doubly for those who, like me, use whitelists for cookies and block third-party content (including cookies) since it generally takes a few attempts to convince the paywall that you just logged in for the umpteenth time and can I now please read that paper please? Nope, thou shalt not pass! aw shucks, I'll just get the thing off Sci-Hub again.
azalemeth 6y ago
I wish sci-hub had a "donate paper" button. I've published things that are open access but the published obsfucate or make hard to find yet get SEO'd to the top of the rankings. I'd love to just upload a pdf, say, of a book chapter.
- henriquemaia 6y ago
  Submit it directly to Library Genesis. There's an upload option there [0] where you can submit scientific papers [1]. Edit: I added some additional info. [0] https://library.bz/main/upload/ https://library.bz/main/upload/ [1] https://i.imgur.com/2YCpTSz.png https://i.imgur.com/2YCpTSz.png
- subroutine 6y ago
  The NIH already has a 'Public Access Policy' for all NIH-funded research projects (which compose the majority of all university research). If only everyone complied... https://publicaccess.nih.gov/ https://publicaccess.nih.gov/ It mandates that within 1 year of journal publication the authors are to make the article available for download on pubmed... "Before you sign a publication agreement or similar copyright transfer agreement, make sure that the agreement allows the paper to be posted to PubMed Central (PMC) in accordance with the NIH Public Access Policy. Final, peer-reviewed manuscripts must be submitted to the NIH Manuscript Submission System (NIHMS) upon acceptance for publication, and be made publicly available on PMC no later than 12 months after the official date of publication."
jillesvangurp 6y ago
Not so surprising. When I was still publishing (until about 10 years ago), I always made sure my papers were easy to find and download. Academic performance is basically measured through scientific references to your work. So basically, a smart scientist would want to do SEO to ensure people can actually find their articles. Even 20 years ago I avoided going to the library to request copies of articles that somehow weren't available online. Nothing gets rid faster of a potentially interested reader than a paywall. I find it surprising, scientists aren't getting smarter about publicizing themselves. All that effort and you can't be bothered to blog about your findings, tweet a bit, engage with your peers online. etc.? There's this notion of spending months years on something and then expecting people to actually find it, pay for it, and then read it only to then consider referring to it. It doesn't work that way if you are just starting out.
nxpnsv 6y ago
Popular papers are more cited! Truly groundbreaking stuff there.
red_admiral 6y ago
From my own experience, putting your papers for free on your discipline's archive (arXiv or whatever), or at least on your personal webpage, is a must if you want citations. There's no excuse for having your work behind a paywall that you don't even get commission from!
zitterbewegung 6y ago
I thought I was good at finding information on Google. Then I took a reading class on quantum computation with a professor. It was like going from a yellow belt to a black belt after two years . I don’t think researchers care that much about rules that hinder them to be publish or perish.
slim 6y ago
Researchers should not be allowed to cite a paper unless they demonstrate proof they actually did pay for the papers they are citing /s
- jkh1 6y ago
  Readers of scientific papers never had to pay. Before papers were available in digital form, you could request a paper copy from the authors. Of course the authors usually paid for some extra copies for this but when those ran out, you'd get a photocopy. Then people started requesting a copy and would get one by email. Now it's just faster to simply find that copy on the web than wait for someone to reply to your email.
akramtariqkhan 6y ago
I have experienced it first-hand due to the fact that referring to paid papers is not possible unless one has a huge budget and the abstract itself isn't enough to decipher what lies beyond!
snicker7 6y ago
Publishers don't care about citations nearly as much as squeezing every last penny from taxpayers that they can (via publically funded research / libraries).
setgree 6y ago
The econometric term for what this paper lacks is "identification strategy" -- a way for assessing the impact of x (downloads on sci-hub) on y (citations) which _identifies_ a source of variation in x that is independent of y. A randomized controlled trial would be ideal. Second-best would be something that, say, suddenly caused some papers to be available or not available on Sci-Hub, or a "plausibly exogenous" (to y) cutoff around which papers would be either available on Sci-Hub or not. This paper offers nothing. I think we can say the relationship is "not identified." Doesn't mean the conclusion is wrong, just that this paper doesn't produce meaningful evidence on the question. For more see Angrist and Krueger (2001) https://economics.mit.edu/files/18 https://economics.mit.edu/files/18 (In Pearl/DAG language, the underlying quality or appeal of the paper is, as others have noted, an obvious confound/backdoor path).
- barmstrong 6y ago
  This is a great summary of how to show causal effects. Thx for pointing it out. I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design). I'm working with a team that is trying to build better tools for science at https://www.researchhub.com/about https://www.researchhub.com/about Rethinking peer review is one item on the roadmap.
  - thomasahle 6y ago
    > we need a new way to do peer review ... so that papers can be upvoted/downvoted This one managed to get 700 upvotes on HN and still be meaningless.
    - Melting_Harps 6y ago
      > I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design). > This one managed to get 700 upvotes on HN and still be meaningless. Agreed. This is why I don't think voting with online peramters, like down voting, will ever be viable choice; because aside from things like Cybil attacks, the reality is that Social Media has normalized and acutely optimized group think and the tribalism that often follow. For those of that don't engage in it its incredibly alarming and disconcerting how many succumb to it's practices in real Life. What I will propose as an alternative is something that was first tried with Andreas Antonpolus' first book 'Mastering Bitcoin' wherein the chapters are each individually uploaded to Github and follows the same process that OSS does, wherein commits and corrections are submitted to alter the books ultimate maintained version, and never really being fully 'finished' and can be amended and annotated as needed to suit the updates that follow (eg: Segwit or Lightening Network) or in this case perhaps a replicated experiment that provides a larger sample size. Replication in Academic Papers is nearly non-existant, despite the notion that Peer Reviewed (especially in STEM) was to be a the critical component that made it invaluable. This could also effectively undo the walled-garden-extrotionist model that afflicts Academia's Peer Reviewed model and foster more interactive and International work without being present in a University. Samples or specimens may need to be more tightly controlled and transported, but this could effectively be done over night if the Will exists for little to no money in anything other than training. If the International University model is undergoing mass disruption and shifting toward a mainly online learning platform, this too could help mitigate these glaring problems and have a Global Repository for ongoing research. Its almost stupid not to do it at this point given the MANY pitfalls that its current model forces down our throats.
aglionby 6y ago
There's some similar work out that analyses the impact on conference paper acceptance of having deanonymised arXiv versions of papers available before review. They look at ICLR papers for the last 2 years. I've not read it in a lot of detail but it looks like there's a positive correlation between releasing papers and having them accepted. Not sure how they've controlled for confounders (you only release papers you're confident in the quality of on arXiv?) https://arxiv.org/pdf/2007.00177.pdf https://arxiv.org/pdf/2007.00177.pdf
Percnopterus 6y ago
There is a long known effect that open access articles get more citations compared to non open access articles (e.g. doi.org/10.1371/journal.pbio.0040157 ). So I do not see why this should be very different for Sci-Hub articles. However I agree that the claimed effect might be exaggerated by confounding. More interestingly proper deposition of the data corresponding with the article increases also the citation rate ( doi.org/10.1371/journal.pone.0230416 )
micelwell0563 6y ago
Same here. Simply put, this is correlation. The irony humorous. https://www.onnewstimes.com/2020/05/fortnite-v-bucks-redeem-v-bucks-gift.html https://www.onnewstimes.com/2020/05/fortnite-v-bucks-redeem-...

754 points by lnyan 6y ago

Appropriately posted on arXiv

Maybe I misunderstood something, but it seems to me their data demonstrate that less interesting papers that nobody bothered to download from SciHub tend to have fewer citations than more interesting ones that are downloaded? How does that support the conclusion “Sci-Hub downloads lead to more article citations” (emphasis mine)? (Yes I read the paper and I’ve seen all the other factors they considered, but those factors simply aren’t as good predictors/indicators of interest as an organic download number from any channel, whether it’s ScienceDirect or SciHub.) Edit: To be clear, papers in their supposedly "limited access" (if we want to draw the same conclusion) control group appear to be available on Sci-Hub, i.e. as open access as their Sci-Hub group. Details in this comment: https://news.ycombinator.com/item?id=23710992 https://news.ycombinator.com/item?id=23710992

Noluris 6y ago
I think the title they chose is just misleading (wrong?). Otherwise, their conclusion is > we found that articles downloaded from Sci-hub were cited 1.72 times more than papers not downloaded from Sci-hub and that the number of downloads from Sci-hub was a robust predictor of future citations > The results suggest that limited access to publications may limit some scientific research from achieving its full impact.
- oefrha 6y ago
  It’s more than just the title. The conclusion from abstract reads “The results suggest that limited access to publications may limit some scientific research from achieving its full impact.” My uncharitable read is they slapped an interesting yet unsupported conclusion onto a rather boring observation. Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim.
  - jaekash 6y ago
    > Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim. I was not sure about this, but if this is the case then their conclusion seems to be unrelated to their findings.
    - oefrha 6y ago
      I'm pretty sure about this now. I just looked into the actual data set[1], and analyzed a few papers from it, e.g. [2] the first paper in scopus_nature.csv. They're definitely available on Sci-Hub at least at the moment [3], and there's no reason to believe they weren't available back then. Also, here's what they say about the dataset, emphasis mine: > As a quality control, we performed a random sampling of all the articles retrieved, excluding those already present in the first data set. As in this second data set, the number of Sci-hub downloads is precisely equal to zero, we regard it as a control group (nC = 4,015) from which we are going to estimate comparisons for our experimental group (nE = 4,646). Downloads equal to zero != not available. [1] https://osf.io/xb9yn/ https://osf.io/xb9yn/ [2] https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141654&partnerID=40&md5=387eaeb70ac807a4fa4acb573e42b03a https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141... [3] https://sci-hub.tw/10.1038/541123a https://sci-hub.tw/10.1038/541123a
      - amatic 6y ago
        > Downloads equal to zero != not available. I'm wondering about this too. What could be the possible other reason for the papers not being downloaded? Boring titles? Preprints easily available? Open access? > and there's no reason to believe they weren't available back then. Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub?
        oefrha 6y ago
        > Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub? That's within the realm of possibilities but sure doesn't sound like the case the way they described their methodology. That would have required a catalog of available papers on Sci-Hub too, but the dataset they used only contained download logs, so they couldn't have known whether a zero-download paper was available or not (unless I misunderstood what the first dataset is about).
- barbegal 6y ago
  I thought that too. But in the "Statistical analyses" section it says: "the interpretation of our robust estimates goes beyond claiming mere correlations or associations." which when you read through the statistical analysis is clearly not true.
- mistaken 6y ago
  I think what they're hinting at is that without SciHub, even a popular paper would have less exposure. Some papers which cite a popular publication can only do so, because they could download the referenced one through SciHub.
  - subroutine 6y ago
    I don't think anyone questions that some articles have been cited because they were downloaded from scihub. This was still a garbage attempt to 'prove' this conclusion. And I don't have any faith the authors measured the true magnitude of these effects.
deleted 6y ago
[deleted]
subroutine 6y ago
Yes this is a huge confound. Their methods state... The first data set contained all articles (nE = 4,646) that were downloaded from Sci-hub between September 2015 and February 2016 [24, 25]. The second data set was extracted from the Scopus database and contained all articles published in the selected journals within the same time period. From this data set, we excluded articles already present in the first data set. Their methods lend to circular reasoning since the conclusion could also be "papers of high interest (cited more often) are more likely to be downloaded from scihub"
chiefalchemist 6y ago
Same here. Simply put, this is correlation. The irony humorous.
Chris2048 6y ago
So basically, correlation, speculated (w/o basis) as causation?
pen2l 6y ago
As someone from a research lab from a big name school that already has access to every journal: we use sci-hub all the time just because it is so damn convenient (me, I have a chrome addon wherein I click a button on a page that has a doi link, and wahlah, I'm looking at the pdf. Firewall at work blocks scihub, but I find ways around it). As to why my lab might be preferring papers from big-name journal: I think 1) there's admittedly our own bias of wanting to cite big papers vs. small papers, 2) if we're doing some basic literature search, Google will also yield results that are already more popular. Really, at the moment, I struggle to see how our behavior changes about which papers we're citing with or without scihub.
- neutronicus 6y ago
  I think without Sci-hub I might have just said "fuck it" and never read certain papers from smaller journals that I later wound up citing One thing I would do a lot with Google Scholar is find a highly-cited, somewhat recent (past decade or so) paper and then click into its citations to find the very most recent related work. IIRC this interface was chronological so there was no big-journal bias and I found a lot of stuff in random journals. Low-friction quality-filtering through SciHub was very helpful for me in this process. I worked in a theoretical field, so it's actually possible to assess the quality of a paper just by reading it. If you're doing something experimental you may have to lean on the big-journal filter a little bit more.
- sitkack 6y ago
  > I struggle to see how our behavior changes about which papers we're citing with or without scihub. Under what conditions do you see that changing? You yourself say how useful it is, but discount the effect it has on the citation graph. Could we say the samething about arxiv? This is exactly why research needs to be open and freely available. The esoteric, rarely cited papers are locked behind paywalls and away from search engines. And in doing so, their impact fades over time as the flock centers around a single vein of focus. The future should look like a net not a river. I am not an academic and I read papers for their metalessons in areas that interest me. But without Sci-Hub, lots of research would not be accessible outside of its niche, esp if it was unlucky enough to not be in the right journal at the right time. On hilarious pattern I see, is when a bigshot in a field publishes a new paper, there is a race to cite it by some nobody. Basically the academic publishing version of "first post".
dekhn 6y ago
I find in nearly all cases, if you invert the causality and the results make more sense, you can ignore a paper. "Papers with more citations are downloaded more frequently on Sci-Hub" makes more sense to me than "Papers downloaded frequently from Sci-Hub get more citations".
credit_guy 6y ago
I guess what you are saying is that correlation does not imply causation, and in this case the causality arrow may go the other way round. In this case, there's something that actually can help establish the direction of the causality arrow: time. If the citations precede the downloads, then sure, it's more well known papers get downloaded more. This papers show that the downloads precede citations though: "number of downloads from Sci-hub was a robust predictor of future citations" While it's not a watertight argument that downloads lead to citations, the theory is at least plausible.

skybrian 6y ago

It seems like downloads from Sci-Hub and citations could have a common cause, for example, being interesting to scientists. If Sci-Hub downloads are a good proxy for interest and they happen first, then they could usefully be used to predict citations, without necessarily being a cause. Different people could be using SciHub versus making citations. I couldn't tell from the paper whether they considered this, or what they think the cause graph looks like.

kgwgk 6y ago
"After controlling for problems of endogeneity, heteroscedasticity, and inconsistency, the interpretation of our robust estimates goes beyond claiming mere correlations or associations." They think they have showed causality. If it only was so easy...
- barbegal 6y ago
  Yeah it doesn't look like they consulted a statistician when doing this research. It always amazes me how a paper like this can be 90% statistics based but non of the authors are statisticians. Without the proper statistical analysis it just becomes a paper on how highly cited papers get downloaded from Sci-Hub which isn't very interesting.
- hatsunearu 6y ago
  Whoever reviews this is probably going to go ballistic lol
  - js8 6y ago
    The reviewers should still hedge their bets and make sure their research is available on Sci-Hub.
fxtentacle 6y ago
There is a very obvious cause, which is that people tend not to cite papers that they have never seen. So if your reviewer tells you that you need more citations, you use the stuff that's easiest to get.

newyankee 6y ago

I mean nowadays i see a lot of motivated and young crowd in India who have become very knowledgeable due to the explosion in many scientific and other such youtube channels in India. Previously amateur researchers could not get access to research papers due to the prohibitive cost of access (unless they are in a really top notch university, which frankly told is highly limited in India as it is purely a numbers game). Sci-hub really enables access to many such folks. A similar story played out in the last decade (2000 - 2010) when affordable streaming platforms did not exist. Many folks got familiar to global movies and shows due to piracy as the price of even legal DVDs were prohibitively expensive.

fxtentacle 6y ago
I think it's also a strong psychological thing. I work for a German company, so paying $40 per paper is no issue. But I still usually close the tab as soon as I see that it's Elsevier. Those people who are interested in me reading their work will send an arxiv link or put the PDF on researchgate or publish open access. Especially for mathematics, I need a printable PDF so that I can take notes. That makes DRMed publications impractical to use. So if someone only links to the paid version of their article, I usually just assume that they're an arrogant prick and skip to the next paper. There's now more good new research being published than I could ever read. Researchers need to adapt to that by reducing friction.
- huijzer 6y ago
  I would like to promote the Netherlands here. The university I'm working for aims to have all papers open-access. This is derived from national policy [1]. [1] https://en.wikipedia.org/wiki/Open_access_in_the_Netherlands#Policy https://en.wikipedia.org/wiki/Open_access_in_the_Netherlands...
- suyjuris 6y ago
  You probably know this already, but for the benefit of others: While it depends on the subfield, a lot of computer science and mathematics papers are freely available. 1. Searching for the title often gives you a PDF in the first few results already. Authors usually have the right to upload a version on their personal website or arXiv. 2. There is Google Scholar [1], which has links to PDFs on the right of the result (if it knows one). It is better at this than regular Google search, in my experience. 3. Manually searching on the author's website (or former website, if they moved) sometimes proves successful, although this is relatively more effort. 4. As mentioned, writing an e-mail to the authors works as well. (If you buy a paper, the authors get nothing, so there is no incentive not to share it.) [1] https://scholar.google.com/ https://scholar.google.com/

PudgePacket 6y ago

https://en.wikipedia.org/wiki/FUTON_bias https://en.wikipedia.org/wiki/FUTON_bias > FUTON bias (acronym for "full text on the Net")[1] is a tendency of scholars to cite academic journals with open access—that is, journals that make their full text available on the Internet without charge—in preference to toll-access publications.

TekMol 6y ago

How does one even use Sci-Hub? The search functionality is "temporarily unavailable" and Google seems to have not indexed the site. Find the abstract elsewhere and then use the DOI to find it on Sci-Hub?

someguyorother 6y ago
>Find the abstract elsewhere and then use the DOI to find it on Sci-Hub? That's pretty much it. Suppose you want to read a paper on memes, so you search scholar.google.com for "memes". Then let's say you find https://www.jbe-platform.com/content/journals/10.1075/etc.10.1.06lou https://www.jbe-platform.com/content/journals/10.1075/etc.10... (I just chose one at random). Going to the link gives you just the abstract. This one happens to have a full text link, but suppose it didn't. Then you'd go to your favorite sci-hub site (e.g. sci-hub.tw) and paste the URL into the "enter URL..." field, and the paper shows up.
grive 6y ago
Yes, pretty much. I find an interesting paper, note its DOI, go to sci-hub to download it. Never tried to search on sci-hub itself.
nisa 6y ago
The few times I've needed it due my uni not having access to a paper using the DOI worked - there is also reddit.com/r/scholar
amatic 6y ago
For me, this website is usually working, with relatively good search http://gen.lib.rus.ec/scimag/ http://gen.lib.rus.ec/scimag/ Though, I often search on scholar and then c/p the title to scihub. Also, http://z-lib.org http://z-lib.org has a good search function, both for books and articles.
eythian 6y ago
I actually quite like using the telegram bot. I just send it a URL or title or whatever and it replies with a PDF.

deleted 6y ago

[deleted]

captn3m0 6y ago

The source for the Sci-Hub downloads is the "Who's downloading pirated papers? Everyone" research from 2016 which was based on anonymized server logs from Sci-Hub. Dataset: https://doi.org/10.5061/dryad.q447c https://doi.org/10.5061/dryad.q447c Research: https://doi.org/10.1126/science.352.6285.508 https://doi.org/10.1126/science.352.6285.508 Article: https://www.sciencemag.org/news/2016/04/whos-downloading-pirated-papers-everyone https://www.sciencemag.org/news/2016/04/whos-downloading-pir...

lowdose 6y ago

Aka the bandwagon effect.

buboard 6y ago

I think the most important is that these citations are being read in full text. This is better that adding a reference based on what it says in the abstract

pedro596 6y ago

It's like saying that if you give free newspapers near the metro it is possible that more people will read them. The only problem is that here they are giving for free "paid newspapers". So, everyone that wanted a paid newspaper but didn't have the money to pay for it read it more times because they were able to steal them. By this analogy the conclusion adds that the fact that people are able to steal newspapers helps to keep everyone more informed. Now, please do the same analogy for food.

guerrilla 6y ago
> The only problem is that here they are giving for free "paid newspapers". Or the problem is that there are "paid newspapers" in the first place. > Now, please do the same analogy for food. No, because information is not a finite resources in the same way that food is: Food can't be cloned at negligible cost after it is produced. If food was replicable like in Star Trek, then at that point we could make the same argument for food. (We do waste a lot of food when people can't pay for it or its transporation and distribution though.)

mmis1000 6y ago

Although they do have positive correlation. Not sure whether it is. The more downloads leads to more citations(so the paper is seen by more people). or The more interesting papers has more downloads(people download papers that are more interesting). Looks like both way makes sense. Not sure which way is contributed more to the correlation?

otherme123 6y ago
I kind of like the other conclusion: Impact Factor of the journal is not associates with the number of citations. IF is an important point in this bussiness. I've been in some job screening in which they aggregate your papers according to the IF of the journals (like one paper in a 1st quartile journal equals three papers of a 2nd quartile). But according to this data you could publish in a 4th quartile that if your paper is interesting, free to download and with some figures, it will be read and cited.

Yetanfou 6y ago

Ghee, whiz, whod'dathunkit that opening up access to papers would lead to more people having access to papers which they'd then use to write more papers in which they cite those papers they read. Even if you have institutional or individual access to the likes of Wiley or Elsevier it is usually far easier to just feed the DOI to Sci-Hub and read the paper instead of jumping through all the hoops to get 'official' access. This goes doubly for those who, like me, use whitelists for cookies and block third-party content (including cookies) since it generally takes a few attempts to convince the paywall that you just logged in for the umpteenth time and can I now please read that paper please? Nope, thou shalt not pass! aw shucks, I'll just get the thing off Sci-Hub again.

azalemeth 6y ago

I wish sci-hub had a "donate paper" button. I've published things that are open access but the published obsfucate or make hard to find yet get SEO'd to the top of the rankings. I'd love to just upload a pdf, say, of a book chapter.

henriquemaia 6y ago
Submit it directly to Library Genesis. There's an upload option there [0] where you can submit scientific papers [1]. Edit: I added some additional info. [0] https://library.bz/main/upload/ https://library.bz/main/upload/ [1] https://i.imgur.com/2YCpTSz.png https://i.imgur.com/2YCpTSz.png
subroutine 6y ago
The NIH already has a 'Public Access Policy' for all NIH-funded research projects (which compose the majority of all university research). If only everyone complied... https://publicaccess.nih.gov/ https://publicaccess.nih.gov/ It mandates that within 1 year of journal publication the authors are to make the article available for download on pubmed... "Before you sign a publication agreement or similar copyright transfer agreement, make sure that the agreement allows the paper to be posted to PubMed Central (PMC) in accordance with the NIH Public Access Policy. Final, peer-reviewed manuscripts must be submitted to the NIH Manuscript Submission System (NIHMS) upon acceptance for publication, and be made publicly available on PMC no later than 12 months after the official date of publication."

jillesvangurp 6y ago

Not so surprising. When I was still publishing (until about 10 years ago), I always made sure my papers were easy to find and download. Academic performance is basically measured through scientific references to your work. So basically, a smart scientist would want to do SEO to ensure people can actually find their articles. Even 20 years ago I avoided going to the library to request copies of articles that somehow weren't available online. Nothing gets rid faster of a potentially interested reader than a paywall. I find it surprising, scientists aren't getting smarter about publicizing themselves. All that effort and you can't be bothered to blog about your findings, tweet a bit, engage with your peers online. etc.? There's this notion of spending months years on something and then expecting people to actually find it, pay for it, and then read it only to then consider referring to it. It doesn't work that way if you are just starting out.

nxpnsv 6y ago

Popular papers are more cited! Truly groundbreaking stuff there.

red_admiral 6y ago

From my own experience, putting your papers for free on your discipline's archive (arXiv or whatever), or at least on your personal webpage, is a must if you want citations. There's no excuse for having your work behind a paywall that you don't even get commission from!

zitterbewegung 6y ago

I thought I was good at finding information on Google. Then I took a reading class on quantum computation with a professor. It was like going from a yellow belt to a black belt after two years . I don’t think researchers care that much about rules that hinder them to be publish or perish.

slim 6y ago

Researchers should not be allowed to cite a paper unless they demonstrate proof they actually did pay for the papers they are citing /s

jkh1 6y ago
Readers of scientific papers never had to pay. Before papers were available in digital form, you could request a paper copy from the authors. Of course the authors usually paid for some extra copies for this but when those ran out, you'd get a photocopy. Then people started requesting a copy and would get one by email. Now it's just faster to simply find that copy on the web than wait for someone to reply to your email.

akramtariqkhan 6y ago

I have experienced it first-hand due to the fact that referring to paid papers is not possible unless one has a huge budget and the abstract itself isn't enough to decipher what lies beyond!

snicker7 6y ago

Publishers don't care about citations nearly as much as squeezing every last penny from taxpayers that they can (via publically funded research / libraries).

setgree 6y ago

The econometric term for what this paper lacks is "identification strategy" -- a way for assessing the impact of x (downloads on sci-hub) on y (citations) which _identifies_ a source of variation in x that is independent of y. A randomized controlled trial would be ideal. Second-best would be something that, say, suddenly caused some papers to be available or not available on Sci-Hub, or a "plausibly exogenous" (to y) cutoff around which papers would be either available on Sci-Hub or not. This paper offers nothing. I think we can say the relationship is "not identified." Doesn't mean the conclusion is wrong, just that this paper doesn't produce meaningful evidence on the question. For more see Angrist and Krueger (2001) https://economics.mit.edu/files/18 https://economics.mit.edu/files/18 (In Pearl/DAG language, the underlying quality or appeal of the paper is, as others have noted, an obvious confound/backdoor path).

barmstrong 6y ago
This is a great summary of how to show causal effects. Thx for pointing it out. I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design). I'm working with a team that is trying to build better tools for science at https://www.researchhub.com/about https://www.researchhub.com/about Rethinking peer review is one item on the roadmap.
- thomasahle 6y ago
  > we need a new way to do peer review ... so that papers can be upvoted/downvoted This one managed to get 700 upvotes on HN and still be meaningless.
  - Melting_Harps 6y ago
    > I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design). > This one managed to get 700 upvotes on HN and still be meaningless. Agreed. This is why I don't think voting with online peramters, like down voting, will ever be viable choice; because aside from things like Cybil attacks, the reality is that Social Media has normalized and acutely optimized group think and the tribalism that often follow. For those of that don't engage in it its incredibly alarming and disconcerting how many succumb to it's practices in real Life. What I will propose as an alternative is something that was first tried with Andreas Antonpolus' first book 'Mastering Bitcoin' wherein the chapters are each individually uploaded to Github and follows the same process that OSS does, wherein commits and corrections are submitted to alter the books ultimate maintained version, and never really being fully 'finished' and can be amended and annotated as needed to suit the updates that follow (eg: Segwit or Lightening Network) or in this case perhaps a replicated experiment that provides a larger sample size. Replication in Academic Papers is nearly non-existant, despite the notion that Peer Reviewed (especially in STEM) was to be a the critical component that made it invaluable. This could also effectively undo the walled-garden-extrotionist model that afflicts Academia's Peer Reviewed model and foster more interactive and International work without being present in a University. Samples or specimens may need to be more tightly controlled and transported, but this could effectively be done over night if the Will exists for little to no money in anything other than training. If the International University model is undergoing mass disruption and shifting toward a mainly online learning platform, this too could help mitigate these glaring problems and have a Global Repository for ongoing research. Its almost stupid not to do it at this point given the MANY pitfalls that its current model forces down our throats.

aglionby 6y ago

There's some similar work out that analyses the impact on conference paper acceptance of having deanonymised arXiv versions of papers available before review. They look at ICLR papers for the last 2 years. I've not read it in a lot of detail but it looks like there's a positive correlation between releasing papers and having them accepted. Not sure how they've controlled for confounders (you only release papers you're confident in the quality of on arXiv?) https://arxiv.org/pdf/2007.00177.pdf https://arxiv.org/pdf/2007.00177.pdf

Percnopterus 6y ago

There is a long known effect that open access articles get more citations compared to non open access articles (e.g. doi.org/10.1371/journal.pbio.0040157 ). So I do not see why this should be very different for Sci-Hub articles. However I agree that the claimed effect might be exaggerated by confounding. More interestingly proper deposition of the data corresponding with the article increases also the citation rate ( doi.org/10.1371/journal.pone.0230416 )

micelwell0563 6y ago

Same here. Simply put, this is correlation. The irony humorous. https://www.onnewstimes.com/2020/05/fortnite-v-bucks-redeem-v-bucks-gift.html https://www.onnewstimes.com/2020/05/fortnite-v-bucks-redeem-...