2 ms·
Why are you blocking PetalBot? It's an actual search engine.
by easrng 5y ago
Why are you blocking PetalBot? It's an actual search engine.
- superkuh 5y agoLegit Huawei IP ranges identifying as Huawei PetalBot were being abusive, definitely not obeying robots.txt, and searching for subsets of content that indicated they were looking to identify political dissidents with no worries about actually indexing the full site. I don't consider it a real search engine. But yeah, maybe not a good fit for this list of educational and copyright parasites.
- easrng 5y agoOh yikes, that sounds bad. It is a real search engine though, https://petalsearch.com/ https://petalsearch.com/