3 ms·
Official CLI Tool for the Internet Archive
- toyg 3y agoI guess I'm one of today's lucky 2000, because I didn't know this existed but the copyright notice says ”2022". Looks useful.
- SushiHippie 3y agohttps://github.com/jjjake/internetarchive/commit/952ace47e0e6a030b76cd0a3fd8f38214bbc3820 https://github.com/jjjake/internetarchive/commit/952ace47e0e... Me too, first commit was a bit more than 11 years ago.
- LeoPanthera 3y agoIt's much older than that, I've been using it for years.
- toomuchtodo 3y agoGithub: https://github.com/jjjake/internetarchive https://github.com/jjjake/internetarchive
- jaboutboul 3y agoGotta love the install a binary via curl instructions.
- _thinx 3y agohaha yes, me too!!! curl is useful, and so popular these day. Always use curl install for tools which I contribute to [1]. [1] https://github.com/docsion/rfsh#install https://github.com/docsion/rfsh#install
- bdcravens 3y agoIt's a Python script. It's also available via Homebrew, but is curl download any worse than Homebrew or npm install? (Obviously curl download piped into "sh" or similar is pretty terrible, but at the end of the day, there's a ton of software out there being installed via trust alone)
- MenhirMike 3y agoIt's super useful because on some very large collections with many files, the .torrent doesn't contain all the files (apparently a long standing bug?), and downloading a file that's dozens of gigabytes big through a web browser requires a lot of prayer and faith that the download doesn't interrupt/can be resumed. So this tool is super handy for that. Heck, it's useful even for small collections just because of how convenient it is.
- renewiltord 3y agoDoes anyone know how much a text-only mirror of the IA is? I would like to keep such a mirror, but I don't want to store the 57 Petabytes it is right now https://wiki.archiveteam.org/index.php/Internet_Archive#Raw_Numbers https://wiki.archiveteam.org/index.php/Internet_Archive#Raw_... The IA's rate limits are quite aggressive (rightly, I suppose) and so I would like to scan my own text-mirror for the precise point I want and then I can click through to the IA's full wayback machine. $10/TiB is about what I'd expect for storage drives and perhaps with the right indexing scheme I can spend $10k or less and have a multi-level cache using overlayfs with goofys or some such equivalent.