3 ms·
Data hoarder that I am I would like it to scrape more of the site than I actually visit ... in case I missed something when I'm browsing offline later.
by JKCalhoun 5y ago
Data hoarder that I am I would like it to scrape more of the site than I actually visit ... in case I missed something when I'm browsing offline later.
- tyingq 5y agoYeah, a "spider this page with depth X" would be a neat addition. Edit: Though the way it works as a sort of proxy, it seems like you could combine it with something like this, and it would just work: https://github.com/naoak/chrome-site-spider https://github.com/naoak/chrome-site-spider
- graderjs 5y agoCool thanks for that, indeed it seems like it would work.
- thricegr8 5y agoWould something like this work for single-page applications? Given that most applications today are built entirely through the DOM, would regex matching on embedded scripts be that useful? I've actually started thinking about how to build a spidering/scraping tool that extracts links/JS, etc from an SPA but would rather not reinvent the wheel if possible.