7 ms·
The US Library of Congress has put 25M items free online
- wordupmaking 9y agoWhat's the copyright? Would it be legal to unzip those and serve them directly, so archive.org or anyone else can make them more inviting for access? I know you shouldn't look a gift horse in the mouth but there's not even an index or a rough idea what something like "Name Authorities" might mean. That's not what I call wide open doors, that more seems like doing some legally required minimum.
- an27 9y agoI suppose you could do that and then wait for the C&D notice? I can't imagine you'd get sued for putting online a copy of this if you comply with C&D notices, then again I'm very narrow-minded. Regarding Name Authorities, this article should clarify it somewhat: https://en.wikipedia.org/wiki/Authority_control https://en.wikipedia.org/wiki/Authority_control It's basically an authentication provider maintained by the Library of Congress, which serves to define cannonical identifiers for library-catalogued entities, like books and public figures. The Library of Congress uses the MARC standard (developed internally) and that is the format of the Name Authorities files: https://en.wikipedia.org/wiki/MARC_standards https://en.wikipedia.org/wiki/MARC_standards
- wonderous 9y agoFiles are 25 million bibliographic index files which were produced by US Federal employees, so yes, they're likely in the public domain as a result. https://en.m.wikipedia.org/wiki/Copyright_status_of_work_by_the_U.S._government https://en.m.wikipedia.org/wiki/Copyright_status_of_work_by_...
- wordupmaking 9y agoAwesome, thanks!
- pmoriarty 9y agoI wonder how much more extensive the release could have been were copyright laws not in the way. Then there's the old question of whether the works under copyright today will ever go in to the public domain, or if their copyright will be extended forever by future changes in copyright law.
- wonderous 9y agoRelease is 25 million bibliographic index files and has nothing to do with copyright, since none of the data was ever covered by copyright protection.
- pmoriarty 9y agoThey didn't have to limit their release just to bibliographic index files. If they wanted to, they could have released manuscripts, letters, newsletters, videos, or any other media they have. But they may have felt inhibited by copyright laws. So my question is, had copyright laws not been an issue, how much more would they have released? There is also the larger question of whether the value of copyright law outweighs the value of not having it, so that everyone can benefit from this treasure trove of knowledge.
- AndrewUnmuted 9y agoI don't think these records meet the standard definition of 'media' anyway. This is really just data that can be used for cataloguing purposes and other media custodian/librarian applications. Given that the LoC has made it their goal to archive at least one copy of everything, I think they are not quite the right people to fall into your anti-copyright cross hairs. However, I do strongly agree with your overall premises.
- gjjrfcbugxbhf 9y agoI don't think the op has anything against the LoC. More like they are lamenting that the LoC has its hands tied.
- Mathnerd314 9y agoCan someone change the title to match the article? s/items/records/
- dogruck 9y agoAnother nit -- paid for via taxes.
- alphonsegaston 9y agoFor those who haven't ever worked with MARC record data before, there's a python library that's a pretty easy interface called pymarc: https://github.com/edsu/pymarc https://github.com/edsu/pymarc
- lgierth 9y agoI added the raw .gz files to IPFS when Library of Congress announced this last week: https://github.com/ipfs/archives/issues/152 https://github.com/ipfs/archives/issues/152 It's slightly more than 100GB and here it is: https://ipfs.io/ipfs/QmWSzgkftVrkh2859bGT44ahzoqcGhFkjsrQUtHen9hVw9/ https://ipfs.io/ipfs/QmWSzgkftVrkh2859bGT44ahzoqcGhFkjsrQUtH... (Note that the filesizes in the directory listing are all wrong -- that's the original index.html from loc.gov/cds/downloads/MDSConnect/) This makes it a lot easier to use this dataset at e.g. hackathons, where a lot of people would simultaneously pester that LoC server, which already seemed pretty bandwidth-limited on its own when I downloaded the files.
- lgierth 9y agoYou can pin it: `ipfs pin add QmWSzgkftVrkh2859bGT44ahzoqcGhFkjsrQUtHen9hVw9` Or list it: `ipfs ls QmWSzgkftVrkh2859bGT44ahzoqcGhFkjsrQUtHen9hVw9` Or copy it into your local filesystem `ipfs get QmWSzgkftVrkh2859bGT44ahzoqcGhFkjsrQUtHen9hVw9`
- toomuchtodo 9y agoDid you push them into the Internet Archive yet? If not, going to grab a beer and start iterating through your IPFS objects.
- lgierth 9y agoGo for it! :) I've never pushed anything to IA so far to be honest.
- toomuchtodo 9y agoFor future reference then! https://github.com/jjjake/internetarchive https://github.com/jjjake/internetarchive
- voltagex_ 9y agoIf someone wants to pay for / donate 100GB worth of bandwidth on a VPS somewhere, I'll do it.
- themodelplumber 9y agoJust in case anyone else is wondering: This is, as I understand it, 25M pieces of metadata, not 25M books, songs, movies, and treasures from the past.
- gt_ 9y agoIt's something along those lines. It looks like the music is all "cover songs" :( one of the heavier metadata types.
- deleted 9y ago[deleted]
- xvilka 9y agoMeanwhile Elsevier, who is widely known for inhibiting science progress by setting incredible high prices even for government funded research papers, makes a move against SciHub [1] and LibGen [2] again [3]. [1] https://sci-hub.cc/ https://sci-hub.cc/ [2] http://libgen.io/ http://libgen.io/ [3] https://torrentfreak.com/elsevier-wants-15-million-piracy-damages-from-sci-hub-and-libgen-170518/ https://torrentfreak.com/elsevier-wants-15-million-piracy-da...
- agumonkey 9y agoI find scihub/libgen a very important project actually. Mirrors should be done (not an order, just a plan).
- petepete 9y agoI've never heard of libgen but it's blocked by my ISP (in the UK)
- user5994461 9y agoIndeed. Access to the websites listed on this page has been blocked pursuant to orders of the high court.
- xvilka 9y agoSciHub has onion site scihub22266oqcxt.onion, I don't know about LibGen though.
- vog 9y agoHere in Germany libgen is not blocked.
- tim333 9y agoOh yeah so it is. https://www.hotspotshield.com/ https://www.hotspotshield.com/ (free) works well for that kind of thing.