5 ms·
A search engine which is not run centrally by one organization on infrastructure in a known network, but rather something like YaCy where individual users run c
by joepie91_ 5y ago
A search engine which is not run centrally by one organization on infrastructure in a known network, but rather something like YaCy where individual users run crawler nodes on networks that vary over time.
Which makes "contact us for an exception" a no-go, as the relevant source IPs will constantly be changing.
- vntok 5y agoYeah, I certainly don't want those crawlers anywhere near my servers. Block by default and allow site admins to unblock should they want to seems like the best way. It is also already how it works with Cloudflare.