3 ms·
Yes, semi-regularly, I did a fresh update since your message. To reduce storage size, only the title/link/metadata of latest post from each feed is saved. I ru
by 8organicbits 2y ago
Yes, semi-regularly, I did a fresh update since your message.
To reduce storage size, only the title/link/metadata of latest post from each feed is saved. I run the crawler manually, aiming for weekly, but sometimes less frequently. So this won't catch every post and it lags behind quite a bit. I'm hitting some hosting sites faster than I'd like, especially ones that support custom domain names, so I'm planning on fixing up the rate-limiting strategy before I put it in a daily cron job.
There's a plan for ArchiveTeam to use the RSS feed as another way of discovering blog posts to archive. I don't think it's generally useful to point your feed reader at it as there's quite a diverse collection of content.