4 ms·
Which companies are we talking about here? In my case the traffic was similar to what was reported here[1]: these are crawlers from Google, OpenAI, Amazon, etc.
by rnhmjoj 1y ago
Which companies are we talking about here? In my case the traffic was similar to what was reported here[1]: these are crawlers from Google, OpenAI, Amazon, etc. they are really idiotic in behaviour, but at least report themselves correctly.
[1]: https://pod.geraspora.de/posts/17342163 https://pod.geraspora.de/posts/17342163
- nemothekid 1y agoOpenAI/Anthropic/Perplexity aren't the bad actors here. If they are, they are relatively simply to block - why would you implement an Anubis PoW MITM Proxy, when you could just simply block on UA? I get the sense many of the bad actors are simply poor copycats that are poorly building LLMs and are scraping the entire web without a care in the world
- rnhmjoj 1y ago> why would you implement an Anubis PoW MITM Proxy, when you could just simply block on UA? That's in fact what I was asking: I've only seen traffic from these kind of companies and I've easily blocked them without an annoying PoW scheme. I have yet to see any of these bad actors and I'm interested in knowing who they actually are.
- whatevaa 1y agoHuawei. Be happy that you haven't been hit by them yet.