12 ms·
Introducing The Paper Bay
- chewxy 14y agoCurious... have you seen /r/scholar? I've requested a few papers from /r/scholar (under throwaways usually), and they've always obliged
- takluyver 14y agoEven within acadaemia, if you don't have access to a paper, it's common enough to ask your friends at other institutions if they can get it for you. We're mostly happy to share knowledge, and we don't feel much solidarity with the publishers who charge £30 for one PDF.
- abdullahkhalids 14y agoInteresting. It would be much more useful if the papers in the 'store' would be visible to everyone to see and request. I wonder though how this ' we will deliver the paper on behalf of that email address.' will be important to challenge any copyright violation claims...
- kriro 14y agoGood luck :)
- buro9 14y agoVery good. But I'd love to have just seen the pirate bay duplicate itself and to create a site exclusively for academic papers. Seeding of obscure papers might be an issue, but I'm sure that it would work quite well.
- lambast 14y agoWhen you download a paper from JSTOR or ScienceDirect, it is quite easy for them to add various watermarks to the PDF, and they already do (mildly). It might make it harder to scale this concept if the provider can track down who initially downloaded the paper (and presumably shared it).
- Create 14y agorequest several incarnations and junk the diff
- jtheory 14y agoThat's fine if you're technical enough to even know what that means; I'm worried about the people working outside the tech realm who could be unaware even of the personal risk when they try to help out.
- kanzure 14y ago> When you download a paper from JSTOR or ScienceDirect, it > is quite easy for them to add various watermarks to the > PDF, and they already do (mildly). You can strip the watermarks with a tool called pdfparanoia. https://github.com/kanzure/pdfparanoia https://github.com/kanzure/pdfparanoia disclaimer: I wrote pdfparanoia because watermarks.
- michaelfeathers 14y agoI never knew that Aaron was in the lineage of the guy behind the Mark Williams C Compiler.
- wglb 14y agoYes. I was working at MWC when Aaron was born. I think I met Aaron once when he was two or three, just briefly. I knew him from stories told by others, and eventually by following his writings. MWC was quite an interesting company in its own right.
- michaelfeathers 14y agoI have fond memories of the compiler. I bought it when my roommate got an Atari ST in college. Very well documented and I learned a lot from the C source of emacs that came bundled with it. I wish I had that source to browse again today.
- wglb 14y agoIt was a very good example of dogfooding--the document reflected what the guys working on the compiler and toolchain would want from such a compiler. I believe the emacs that you are talking about is MicroEmacs, written by Dave Conroy while working at MWC. The source has since been made available generally, and the editor is the one that Linus uses. Daniel Lawrence later took over the distribution and here is one location: http://www.aquest.com/emacs.htm http://www.aquest.com/emacs.htm I used it for quite a while, but then fell into Emacs and didn't look back. Oh--I did get it to run on my HP200lx and used it there for a while.
- stared 14y agoWhile I agree with the "manifesto", the solution seems to be not that much radical. There are many places where people can upload their papers. Moreover, a web1.0 interface does not seems to be efficient. In that line there is already: http://www.pirateuniversity.org/ http://www.pirateuniversity.org/ And... judging by it's name (i.e. The Paper Bay) I would expect something much more radical, actually aiming at getting 75TB of _any_ paper content...
- Create 14y agoperhaps it should have magnet links and torrent files like archive.org and an API to upload, .bib-s and RDF-s in the url bar for the likes of zotero. http://arxiv.org/help/api/index http://arxiv.org/help/api/index http://archive.org/about/faqs.php#Archive_BitTorrents http://archive.org/about/faqs.php#Archive_BitTorrents
- axusgrad 14y agoThe site does not host or link to copyrighted work. I think he thought of a way to make this happen without breaking the law (himself).
- devopstom 14y agoThis is a bit of a grey area that we deliberately avoided with PDFTribute.net; it seems perfectly acceptable to aggregate and index metadata, just as long as you're not actually storing any data.
- brownbat 14y agoAlready featuring a request for "cocaine," no author given. Relying on the benevolence of others may only go so far...
- jacquesm 14y agoNot any more... Of course the 'abuse angle' was taken into account, it's a bit annoying but any user facing service will have that aspect.
- roel_v 14y agoWith the exception of old and obscure papers (i.e., 15 years or more and not cited in the last 5 years), I don't really see what the big problem is. I have yet to mail an author and not receive a pdf of the paper, usually within days, sometimes within minutes. When you have a reference and Google, you're seconds away from finding that author's current affiliation & email address. I once emailed the author of a book that retailed for 150$+ on some Elsevier subsidiary to ask for a few pages. Turns out the book was basically her PhD thesis (many, many books are like that) and she send me the pdf of that thesis a few hours after I emailed her. All this brouhaha over access to papers is a storm in a glass water if you ask me, instigated mostly by those who don't even need the papers but rather are looking for a big meany to pick an ideological fight with. Maybe it differs per field, I don't know.
- stared 14y agoGood luck with emailing authors of papers which are 70 years old. In mathematics good results never outdate, so there is no rule of thumb "if it is still relevant, author must be still alive".
- roel_v 14y agoBut those perennial papers are usually so widely spread that a simple google search will bring them up, or otherwise somebody in the department will have a copy on their hard disk; so finding it is as simple as asking 'hey who has xyz' on the internal mailing list. Maybe it's different in mathematics, I don't know, I'm just stating my experiences. Few fields are as long-lasting as mathematics, too, I think; in most fields, only a hand full of papers is still relevant after 20 years.
- stared 14y agoIt isn't as simple, as a perennial paper in mathematics does not have to be even famous. It may have even not cites (yet). And I'm speaking of my experience (theoretical physics). Moreover, then you can hardly expect that one of a bunch of guys at your department has the paper. Sure, for applied (or even - experimental) sciences it is different.
- handycapo 14y agoGet this on Tot/i2p. It'll never go down...
- p4bl0 14y agoI agree with you on Tor and/or i2p. I don't know if you can be as categorical as "It'll never go down", but at least it will be significantly harder to get it down. And it can be anonymous, which is important of course for the people offering the service, but also to protect the users of the service… I don't know if such a website can survive to the scientific publishers' lobby, especially when its builder is publicly known. Edit: it seems this exists, as another comment point out on this discussion page.
- famo 14y agoNice idea. Perhaps evolving this to include raw research data could be interesting... e.g. may I please have a) your research paper and b) the 20Gb of trial data you used to come to your conclusions. With this I'll run my own experiments, expanding on your trials and hopefully adding to the wealth of knowledge you so wonderfully kick started. Cheers.
- famo 14y agoYeah sure, here you go. Sincerely, Ms Kate Awesome, Phd
- revorad 14y agoGreat effort! Please add https://peerj.com/ https://peerj.com/ and http://www.mendeley.com/ http://www.mendeley.com/ to your list of resources.
- jacquesm 14y agoI'll add those tonight, thanks! Ok. done.
- mHORxaq 14y agoI submitted articleak, which something very similar a few days ago, but it didn't get upvoted. Maybe people on Hacker News do not like Tor? The idea of using Tor is that the site is way harder for authorities to get down, and even if they do break Tor and get to the server, the users will still be entirely protected (there can't be any information about them in the logs or anything). link: http://articleak.allalla.com/ http://articleak.allalla.com/ submission: http://news.ycombinator.com/item?id=5175234 http://news.ycombinator.com/item?id=5175234
- SkyMarshal 14y agoWhether you get upvoted or not has as much to do with time of day of submission and random factors as how much people like it.
- jpallen 14y agoCould you allow the requester to provide a link to the pay wall where they get stuck? People who have access to the paper through a university subscription could then easily click on that link and then immediately be taken to the paper. I imagine most people looking for a paper will already have come across this paywall and so will know where to find it. It would significantly reduce the overhead of the person fulfilling the request, but may be against the spirit of the site. In many cases that I have come across, the DOI can be turned into a link that takes you to the article. Perhaps that would be enough.
- kerneis 14y ago> In many cases that I have come across, the DOI can be turned into a link that takes you to the article. Perhaps that would be enough. This would be a very simple but extremely helpful change. You only need to prepend http://dx.doi.org/ http://dx.doi.org/ in front of the DOI, and make the resulting link clickable.
- jacquesm 14y agoOk, I'll add that tonight. edit: done.
- mHORxaq 14y agoYou could also add the various other efforts in Open Access domain that were linked on this page to your http://thepaperbay.com/others/ http://thepaperbay.com/others/ page. There was at least Pirate University, articleak, and /r/scholar. Also, while it's okay to have a somewhat complexe form to fill when making a request, it's really sad that it's as much work to share a paper, it should be very easy, in particular when clicking "I have this, send it", the form should have only one field: the file upload, or at least have the other prefilled with the request's values.
- Matti 14y agoThe filling a request process appears to be more complex than it is due to superfluous form fields. The information about authors, title and DOI has already been entered by the requester. For the service to be a viable alternative, it should be (and appear to be) extremely simple to fill a request. Tracking down a paper is time consuming enough, even if you are sitting on the network with access to most journals. Don't make it look like people have to type in additional information manually.
- mHORxaq 14y agoAt articleak (see my other comment), we only ask for the link to the paywall, and then if we could not get it from there (we make the http request via Tor for obvious reasons so it may fail some times) we ask you for the title of the paper, and that's all. So in the worst case you have to fill in two fields, but in most case, only one: the link to the paywall. However, I would not be as hard as you seem to be with the pirate would made The Paper Bay. When you really need a paper and you already lost half an hour searching for it in the web ocean, it's okay to take 2 more minutes instead of 30 seconds to ask for it on such a service.
- joelthelion 14y agoThanks a lot, this is very nice and helpful. However... Aren't you afraid of going to jail?
- deutronium 14y agoI'm wondering if you could do something like this automatically, so users with access to journals could download a little program, and then other users could send queries via a P2P program, which would end up at nodes with access to journals. There could be various settings to ensure each node with access to journals throttle the amount of downloads.
- djt 14y agoIs this legal?
- djt 14y agoIt's just a question guys, not a moral judgement.
- jaipilot747 14y agoWhat is stopping the server from being taken down by legal pressure?
- TimReynolds 14y agoI got bored reading the post so started to look around the website. I think the Domains for Sale page (http://www.jacquesmattheij.com/auction-of-domains-for-sale/ http://www.jacquesmattheij.com/auction-of-domains-for-sale/) only clarifies my initial thoughts on this person.
- user24 14y ago... how does selling off unused domains relate to this at all?
- p0ppe 14y agoOr you could have a look at this page instead; http://jacquesmattheij.com/thank-hn-our-friend-is-safe-and-sound http://jacquesmattheij.com/thank-hn-our-friend-is-safe-and-s...
- Lambdanaut 14y agoI hear he also picks his nose! I don't care if the guy smuggles cocaine on the side, this specific thing he's doing seems like a really useful service.
- dia80 14y agoFirst request and got the paper already!
- popmilo2 14y agoMy first request also and got the paper after an hour ! Who said, good people are rare to find :)
- twodayslate 14y ago"Store" makes it sound like I should be buying something. Perhaps change the name to "Upload"
- cllns 14y agoIt's a bit confusing that the site (ThePaperBay.com) has the same theme as your blog.
- Jasber 14y agoThis is because they're both using http://octopress.org/ http://octopress.org/
- speednoise 14y agoThat doesn't really make it any less confusing
- mr_luc 14y agoYou're a really good guy, jacques. And I had no idea about Mark Williams C.
- fwr 14y agoWe have a thriving, helpful community already up and running at http://www.reddit.com/r/Scholar/ http://www.reddit.com/r/Scholar/.
- pepr 14y agoI kind of hoped for an actual torrent tracker rather than central site that can be easily taken down. But never mind :)
- pla3rhat3r 14y agoThe UI is not very clean but I love the idea. I wish there was a browse feature. Some of us that aren't in college just enjoy enlightening our brain and would love to see what's already up on the site.
- groby_b 14y agoThat's a great idea - but way to bury the lede there. (And it just occurred to me that that's journalisms way of saying tl;dr. Except I did read it, just to find what the heck The Paper Bay is :)
- megablast 14y agoI have trouble with someone who is also a domain squatter: http://www.jacquesmattheij.com/auction-of-domains-for-sale/ http://www.jacquesmattheij.com/auction-of-domains-for-sale/
- wicknicks 14y agoReddit has a channel for paper requests. Might be a useful place for 'much needed features' http://www.reddit.com/r/Scholar http://www.reddit.com/r/Scholar
- mikevm 14y agoI guess this will come in handy: https://github.com/kanzure/pdfparanoia https://github.com/kanzure/pdfparanoia
- ivan_ah 14y agoYour article is very well worded. I like the idea, but I am not sure if it is reasonable for people "on the inside" to spend time fulfilling requests. Could there be a js solution to this? A volunteer inside the paywall could leave a tab open which periodically goes to get requests from the main paperbay queue and tries to fulfill them (you would need some robot-like functionality like find the PDF link). If success, it uploads the PDF. If fail (no subsc?), it can notify paperbay to put the request back in the queue. With one requests every 10 minutes and 1000 volunteers you could have a 6000 paper/ hour rate of exodus. BTW? Where are you hosted? How long do you think before they come for you if it gets big? ________________ PS: The protocol could be extended a bit it we need to handle captchas' as well. These really piss me off because I can't get the paper via ssh to campus + elinks! R: requestor F: friend (inside paywall) R-->tpb.com doi:10000x200 PLZ tpb.com: resolve doi:10000x200, prepare scrape recipe. tpb.com-->F could you get jrnl.com/yr/issue/33131/ F-->jrnl.com GET ... F<--jrnl.com CAPTCHA.jpg + form el R<--tpb.com<--F solve plz ( CAPTCHA.jpg , form el ) R-->tpb.com-->F form ans F-->jrnl.com captcha form submit R<--tpb.com<--F<--jrnl.com PAPER.pdf