4 ms·

Why can't crawling be crowd sourced? It would solve ip rotation and spread the load

• Poomba 1y ago
  That’s how residential proxies work, in a perverse way
• 6510 1y ago
  https://yacy.net https://yacy.net
  • catlikesshrimp 1y ago
    Too bad it doesn't support android. It is much more energy efficient than anything else I can spare (for 100% uptime contribution)
• deleted 1y ago
  [deleted]
• chiefsearchaco 1y ago
  Common crawl sort of serves this function. I use it. It's a really good foundation.