14 ms·
Sci-Bay: Google Scholar plus Sci-Hub
- deleted 9y ago[deleted]
- bringtheaction 9y agoJust tested it by searching for “spline”. This is great! Can someone elaborate on how it was made? Specifically, how it integrates with Google Scholar. Is that done client side or server side? If server side how come it hasn’t been blocked by Google seeing as Google don’t seem to like robots using their regular search function so my guess would be they wouldn’t like it for Scholar either. Perhaps it is proxying requests and would also pass on any CAPTCHAs presented? Still in that case I would expect all requests to get hit with a CAPTCHA. Perhaps it just hasn’t had enough traffic yet?
- k5hp 9y agoInteresting that Hacker News allows links to pirate sites... This will probably get removed soon
- veridies 9y agoI wouldn't think Hacker News would be in any trouble with this one. The link is to a noteworthy site, not to any infringing content; you'd have to take action and use a specific search term to find copyrighted content. And the implication of posting it is clearly that people should find it interesting and discuss it, not that they should use it to download articles.
- zeth___ 9y agoAnd it's not like searching in google scholar, getting the doi and putting it in scihub was any great hardship before.
- pbhjpbhj 9y agoA guy in the UK got extradited to USA for making a web page with links to sites that hosted copyright material ... YMMV. https://www.techdirt.com/articles/20120113/09184917400/us-to-extradite-uk-student-copyright-infringement-despite-site-being-legal-uk.shtml https://www.techdirt.com/articles/20120113/09184917400/us-to...
- petra 9y agoHow did you integrate scholar in sci-bay ? Does scholar have an API ? And about the future, how do you see google responding?
- shakna 9y agoI don't know how they are doing it, but Google Scholar does not have an API, and scraping is against their TOS. > Don’t misuse our Services. For example, don’t interfere with our Services or try to access them using a method other than the interface and the instructions that we provide. Despite this, there is scholar.py [0], which can extract files from Google Scholar, though it explicitly doesn't work around the rate limits. [0] https://github.com/ckreibich/scholar.py https://github.com/ckreibich/scholar.py
- crispyporkbites 9y agoHttp is an interface with implicit instructions (especially if restful), provided by google
- userbinator 9y agoor try to access them using a method other than the interface Unless this actually exploits something and hacks into Google's servers to get to the content, which would be something quite different, it wouldn't really be distinguishable from someone manually visiting the site in a browser, volume aside. IMHO the pervasive attitude today of somehow requiring permission or an explicitly sanctioned "API" to access what is otherwise publicly accessible data is rather troubling for the freedom and flexibility of the Web as a whole. It encourages walled-garden content models and centralisation.
- shakna 9y agoI absolutely agree. If something is publicly accessible then the public should be able to use it as they see fit, from my viewpoint. (A HTTP response has already authorised you to copy the data to a machine. How can it be bound by a TOS that you need to access the original page to find?) However, Google doesn't agree and the current court precedent doesn't either. So I tried to address the parent's concern from that viewpoint.
- jrochkind1 9y agoGoogle Scholar definitely and intentionally offers no API. I don't see this lasting long...
- mchannon 9y agogoogle.com/scholar doesn’t work for you?
- vha32oiwe 9y agoThey called out API specifically Not UI
- wyldfire 9y agoThe issue is regarding: this service (Sci-Bay) depends on Google Scholar, yet there's no public API for Google Scholar that it could leverage. If it's scraping Google Scholar results, then it's likely a ToS violation and unlikely to last long.
- userbinator 9y agoHow much of a "ToS violation" is SciHub, and how long has it lasted? "If there's a will, there's a way" comes to mind. Also, the fact that all web pages technically already have an "API" --- it's called "HTTP" ;-) Good for them, I say.
- gpm 9y agoAt a glance it looks like it's really just a proxy, that was limited to scholar.google.com and mutates the page slightly (adds a header, sci-hub links). Does google generally block proxy servers?
- jrochkind1 9y agoI don't know if they generally do, but I'm sure they can/will if they want to.
- Spivak 9y ago
- jmnicholson 9y agoHow is this any better than using the sci-hub plugin?
- agumonkey 9y agoreminder that this went up not long ago https://whereisscihub.herokuapp.com https://whereisscihub.herokuapp.com
- bcaa7f3a8bbc 9y ago> If sci-hub is going to scrape publishers, they could put in a bit more effort. +1, especially for the Onion site. Onion service supposed to be a primary mean to host uncensored websites instead of having to look for the latest domain name everyday, unfortunately it seems nobody cares about it. Most of the time I access it from my browser, the front-end proxy was malfunctioning, or the back-end Tor daemon has dead... Tor network itself do have capacity problem, but they could do much better than a broken front-end proxy... e.g. with Onion Balance.
- Vinnl 9y agoThanks for sharing; note that I've moved that to a somewhat simpler URL: https://whereisscihub.now.sh/ https://whereisscihub.now.sh/ I should also add that I am also working on a project to incentivising authors to make their work freely available: https://flockademic.com/ https://flockademic.com/ (More info here: https://medium.com/p/the-holy-grail-in-open-access-sharing-that-benefits-authors-b685ff2c6300 https://medium.com/p/the-holy-grail-in-open-access-sharing-t... )
- Aelius 9y agoSci hub seems like the perfect candidate for ipfs, I'm astounded the mirrors haven't implementated that yet.
- JustFinishedBSG 9y agoWe just need decentralized DNS for sci-hub, storage isn't needed (right now). Plus sci-hub storage requirements are pretty big, >75 Tb...
- zaarn 9y agoA dataset at the scale of Scihub would have to pay someone to keep all the various PDFs online (which IIRC is around 70TB by now), which would mean a DMCA or N&T or similar would take down paid IPFS hosters. I'm not sure how many people would willingly play IPFS hoster for 70TB datasets for free and while not giving a shit about authorities knocking on the door.
- eruci 9y agoIt does not work.
- stevespang 9y agoSci Hub is best, no games - - just the real deal.
- tomrod 9y agoOh, this is wonderful!
- fwgwgwgch 9y agoCan someone help dispel my ethical concerns over using papers like this? Eg sci-hub. Any and all arguments, on both sides are very welcome.
- lvs 9y agoWell, the research and the researchers are all almost entirely funded with public funds from around the world. The fact that they are forced for professional reasons to funnel all their work into a for-profit publishing industry for distribution is a historical anachronism that dies hard. In a broad sense, there's only very weak analogy to music or film piracy in terms of the process of creative work. Everyone has already paid for scientists to do this science. Scientists and institutions have even paid the journals to publish their work in almost all cases, and they won't reap any reward if you read the work through the legitimate distribution channel. You only pay the publisher, not a starving researcher. Personally, I'd feel very happy if someone read my papers by any means necessary. That was the whole point of writing them, setting aside the practical annoyance of simply trying to keep my career afloat. Edit: The lifetime value of a single paper to the journal can actually be priced, since they started charging researchers for open access a few years ago (often in addition to other fees). This number is usually in the $2-5K range per paper, in my field.
- theptip 9y agoA timely request: http://slatestarcodex.com/2018/03/19/the-dark-rule-utilitarian-argument-for-science-piracy/ http://slatestarcodex.com/2018/03/19/the-dark-rule-utilitari... "So this is my argument that Sci-Hub can be ethical. Universalized it would destroy the system – but the system is bad and needs to be destroyed. And although this would break the law, a very slight amount of law-breaking might be a beneficial solution to inadequate equilibria that could be endorsed even when universalized."
- lsh 9y ago> but the system is bad and needs to be destroyed The system needs to be reformed, and that is currently happening, peacefully. Perhaps not everywhere at once and perhaps not as quickly as you might prefer, but open access publishing has made great strides in just a few short years. This doesn't liberate the millions (probably) of academic articles whose authors relinquished their copyright to big publishers, but it is resulting in new tensions like Germany and Elsevier butting heads (https://www.nature.com/articles/d41586-018-00093-7 https://www.nature.com/articles/d41586-018-00093-7) and SciHub and exciting new legislation No destruction or 'very slight law-breaking' necessary, nor mealy-mouthed 'inadequate equilibria' lies-to-self necessary either. Change is happening, the world doesn't need to go all Mad Max.
- lihan 9y agoWe'll see how long this one last.
- Tomminn 9y agoHoneypot?
- Myrmornis 9y agoI'm 100% in favour of sci-hub. However, note that they are very anarchic when it comes to commercial books, not just journal articles! E.g. from the Sci-Bay search results, this is $131 on amazon.com, and quite possibly the authors do want the royalties. [BOOK] Intelligent optimisation techniques: genetic algorithms, tabu search, simulated annealing and neural networks D Pham, D Karaboga - 2012 - books.google.com ... Cited by 916 Related articles All 3 versions [Download Book]
- gkya 9y agoI believe people in academia are paid to write these books anyways, so they might as well not receive the royalties. As a prospective academic myself, I find it unethical.
- dagw 9y agoI believe people in academia are paid to write these books anyways While that does occasionally happen, it's definitely not true in general. The people I know who wrote academic books did so by taking a sabbatical from their university job and/or working evenings and weekends for the actual writing. Occasionally they can apply for a separate writing grant to cover their lost salary, but that's completely separate from their day job.
- gkya 9y agoWhat I meant was their job is to author such books. Not that they should be paid separately for them. Depending on the country academicians might be underpaid, but that's an issue on its own.
- dagw 9y agoWhat I meant was their job is to author such books Except in many cases it isn't. Most of the people I know who have written academic books did so off the clock and on their own time. Sure the university let them use their university office and resources, and obviously much of the research the book is based on is research they'd already done as part of their job, but the actual writing time and any additional research they had to finance from sources unrelated to their university job.
- hui630811868 9y agoSebs
- xstartup 9y agoIt works well, OT: Anyone knows how to remove the top header in this Scihub link: https://sci-bay.org/article?link=https://pdfs.semanticscholar.org/88d6/33703f6c58c54a3dc8140767b9fd7ae19ed2.pdf&info=simZbOimsdwJ&scirp=2&k=a&pd=https://pdfs.semanticscholar.org/88d6/33703f6c58c54a3dc8140767b9fd7ae19ed2.pdf&citn=105&cit=15902675276406532530 https://sci-bay.org/article?link=https://pdfs.semanticschola...
- gpm 9y agoDownload the pdf, open it in your browser (or another pdf reader) directly.
- lsh 9y agoIf sci-hub is going to scrape OA publishers, they could put in a bit more effort. For example this (which sucks): https://sci-bay.org/article?link=https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4559886/&info=T5ySYQkNqxIJ&scirp=0&k=a&pd=&citn=176&cit=1345183247643089999 https://sci-bay.org/article?link=https://www.ncbi.nlm.nih.go... Versus the actual article: https://elifesciences.org/articles/24234 https://elifesciences.org/articles/24234
- deleted 9y ago[deleted]
- n4r9 9y agoI think the issue is with sci-bay rather than sci-hub. Searching sci-hub for the title of the article brings you to the second webpage you linked.
- PokemonNoGo 9y agoI'm sorry but I don't see how it works? >https://sci-bay.org/scholar?hl=en&as_sdt=0%2C5&q=entropy+shannon&btnG= https://sci-bay.org/scholar?hl=en&as_sdt=0%2C5&q=entropy+sha... -> Please show you're not a robot
- samat 9y agoThis reminds me of Popcorn time so much. Rightholders do not fear torrents as long as they are unusable for the general population. The second they see something usable — they go berserk. Gonna need some popcorn to watch this one.
- Vinnl 9y agoLikewise, the traditional publishers often respond to demands by funders to make research available by e.g. allowing researchers to share their work elsewhere, and often only after a year or so after publication [0]. This makes the barrier to do so higher, and makes the research less findable. It's not odd to expect that when initiatives like Unpaywall [1] make that research more discoverable, things like embargo periods will get worse. [0] https://medium.com/flockademic/how-open-can-open-access-be-cf6662565ecd https://medium.com/flockademic/how-open-can-open-access-be-c... [1] https://unpaywall.org/ https://unpaywall.org/
- moomin 9y agoThis is a clever mashup. Of course, if you want it to last a week, I'd be making some effort to distribute the source far and wide...
- rkskejfj 9y agoBeyond close vs. distant ties: Understanding post-service sharing of information with close, exchange, and hybrid ties
- c13564 9y agojaeyoung cho
- irundebian 9y agoAaaaand it's down.
- deleted 9y ago[deleted]
- s2th4d 9y agoAnd it's down. "See you later Too much attention is a bad thing, Sci-Bay decides to stop service for a while. Sorry. Anyone who knows how Sci-Bay works and wishes this tool benefits more academics, please contact: info@sci-bay.org"
- aysus 9y agoBarOn, R. (1997). EQ-i Baron Emotional Quotient Inventory: A Measure of Emotional Intelligence : Technical Manual. Toronto, ON: MHS.
- abhishekjha 9y agoIs the website down? It just says "too much attention" caused it to shut down.