3 ms·

Here's another self-promotion. https://alexsci.com/rss-blogroll-network/ https://alexsci.com/rss-blogroll-network/ This uses OPML blogrolls to crawl blog-to-b

Here's another self-promotion. https://alexsci.com/rss-blogroll-network/ https://alexsci.com/rss-blogroll-network/ This uses OPML blogrolls to crawl blog-to-blog recommendations. I seeded it with the blogs I follow and various planets (https://indieweb.org/planet https://indieweb.org/planet) and then recursively followed recommendations to build an organic network. Lots of the content is tech-related, indieweb, and smallweb. It's grown to 17 languages and over 4000 RSS/atom feeds. As an example, the linked blog has a page here [1] and it was discovered by a recommendation by [2]. [1] https://alexsci.com/rss-blogroll-network/discover/feed-12ac5c19b1de9ffa432f0063f84539c4/ https://alexsci.com/rss-blogroll-network/discover/feed-12ac5... [2] https://alexsci.com/rss-blogroll-network/discover/feed-8ecf912379244d536a74e8070a923b6e/ https://alexsci.com/rss-blogroll-network/discover/feed-8ecf9...

freediver 2y ago
This is cool! Is the aggregate list supposed to update regularly? https://alexsci.com/rss-blogroll-network/rss.xml https://alexsci.com/rss-blogroll-network/rss.xml
- 8organicbits 2y ago
  Yes, semi-regularly, I did a fresh update since your message. To reduce storage size, only the title/link/metadata of latest post from each feed is saved. I run the crawler manually, aiming for weekly, but sometimes less frequently. So this won't catch every post and it lags behind quite a bit. I'm hitting some hosting sites faster than I'd like, especially ones that support custom domain names, so I'm planning on fixing up the rate-limiting strategy before I put it in a daily cron job. There's a plan for ArchiveTeam to use the RSS feed as another way of discovering blog posts to archive. I don't think it's generally useful to point your feed reader at it as there's quite a diverse collection of content.

freediver 2y ago

This is cool! Is the aggregate list supposed to update regularly? https://alexsci.com/rss-blogroll-network/rss.xml https://alexsci.com/rss-blogroll-network/rss.xml

8organicbits 2y ago
Yes, semi-regularly, I did a fresh update since your message. To reduce storage size, only the title/link/metadata of latest post from each feed is saved. I run the crawler manually, aiming for weekly, but sometimes less frequently. So this won't catch every post and it lags behind quite a bit. I'm hitting some hosting sites faster than I'd like, especially ones that support custom domain names, so I'm planning on fixing up the rate-limiting strategy before I put it in a daily cron job. There's a plan for ArchiveTeam to use the RSS feed as another way of discovering blog posts to archive. I don't think it's generally useful to point your feed reader at it as there's quite a diverse collection of content.