3 ms·
It occurs to me again that I need to figure out how entire websites can be downloaded and archived. Like Archive.org, but local.
by anononaut 3y ago
It occurs to me again that I need to figure out how entire websites can be downloaded and archived. Like Archive.org, but local.
- psychoslave 3y agohttps://bash-prompt.net/guides/wget-mirror-website/ https://bash-prompt.net/guides/wget-mirror-website/
- danparsonson 3y agohttps://www.httrack.com/ https://www.httrack.com/ is a good option
- mesarvagya 3y agoMost of SPA websites today cannot be downloaded through httrack.
- danparsonson 3y agoYes SPAs will be next to impossible for a tool like this; I'm not sure how any tool could archive such a site tbh?
- _a9 3y agoI use browsertrix-crawler[0] for crawling and it does well on JS heavy sites since it uses a real browser to request pages. Even has options to load browser profiles so you can crawl while being authenticated on sites. [0] https://github.com/webrecorder/browsertrix-crawler https://github.com/webrecorder/browsertrix-crawler
- shagie 3y agoThe entirety of the archive pre rework can be found at https://archive.org/details/c2.com-wiki_201501 https://archive.org/details/c2.com-wiki_201501