6 ms·
> Note: This website will be shut down on December 31, 2024, so if you want to download anything, don’t wait too long. I hope this collection gets onto archive
by pabs3 6y ago
> Note: This website will be shut down on December 31, 2024, so if you want to download anything, don’t wait too long.
I hope this collection gets onto archive.org.
- kwhitefoot 6y agoDon't just hope. Help to put it there. Either paste urls into the Wayback Machine [1] or use the Firefox extension to add the pages to the Wayback Machine. I've added some. Don't forget to add the actual pdfs. [1] https://web.archive.org/save https://web.archive.org/save
- Rebelgecko 6y agoDoes that matter if they just add the whole website to their robots.txt in December of 2024?
- pabs3 6y agoIIRC archive.org pay attention to the robots.txt now rather than in 2024.
- moonchild 6y agoThey do not. They have in the past removed public access to pages due to robots.txt changes.
- pabs3 6y agoHmm, this old blog post sounds like they were planning on ending that practice: https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...
- pabs3 6y agoI just used savepagenow@archive.org to archive all the web parts and then a bunch of curl plus the email to save the PDFs.
- toomuchtodo 6y agoThank you. There is also a small Python app to kick off archival requests if the CLI is more your thing (I find it helpful when I need to generate a list of URLs and then pipe that list to savepagenow with a sleep in between runs to be gentle on the Wayback API). https://github.com/pastpages/savepagenow https://github.com/pastpages/savepagenow
- pabs3 6y agoI think I prefer to use the savepagenow@archive.org email address, best leave it up to archive.org to schedule the archivals. Its also more convenient to paste some HTML into an email and have all the links archived.
- schwartzworld 6y agohttps://schwartz.world/blog/archive_bookmarklet/ https://schwartz.world/blog/archive_bookmarklet/ I found this bookmarklet code here on HN
- pabs3 6y agoThat seems to be for archive.is rather than archive.org.