2 ms·

It’s effective against teenagers maybe. Not so much against Amazon, Meta or wherever botnet/crawler is coming out of China these days from up-and-coming AI comp

by Jolter 5mo ago

It’s effective against teenagers maybe. Not so much against Amazon, Meta or wherever botnet/crawler is coming out of China these days from up-and-coming AI companies.

tonetegeatinst 5mo ago
I mean you could block entire AS numbers that relate to amazon or big tech datacenters
- tangledhelix 5mo ago
  wouldn't help, much of the traffic we've observed look closer to ddos patterns - IPs from all over the world, many different networks, each IP makes one request only, doesn't come back. highly distributed, no form of blocking would be effective except maybe captcha or proof of work.
lucb1e 5mo ago
Then block all of Amazon, Meta, or wherever botnet/crawling traffic is coming from that doesn't honor robots.txt, sends DDoS reflection traffic, submits SMTP messages (in large volumes, not just probing) for domains they're not authorized for with SPF, or whatever else applies to the protocol you're using If they can't keep their ranges clean to a reasonable degree, their customers will need to move if they want to access your part of the internet. New sign-ups will always be hard, so some amount of abuse is expected, but if it's the same abuse traffic for weeks after you've notified them, well, it stops being your problem at some point
- Jolter 5mo ago
  See the other comments in this thread. The perpetrators are unknown and are jumping between residential IPs. Possibly botnets?
  - lucb1e 5mo ago
    Then see my other replies in the thread where I've specifically addressed residential IPs, e.g.: https://news.ycombinator.com/item?id=48163060 https://news.ycombinator.com/item?id=48163060
    - Jolter 5mo ago
      This is the post I’m talking about. Make sure you understand how it would not be productive to go after each ISP individually when the traffic is from all of them. https://news.ycombinator.com/item?id=48155512 https://news.ycombinator.com/item?id=48155512

by Jolter 5mo ago

It’s effective against teenagers maybe. Not so much against Amazon, Meta or wherever botnet/crawler is coming out of China these days from up-and-coming AI companies.

tonetegeatinst 5mo ago

I mean you could block entire AS numbers that relate to amazon or big tech datacenters

tangledhelix 5mo ago
wouldn't help, much of the traffic we've observed look closer to ddos patterns - IPs from all over the world, many different networks, each IP makes one request only, doesn't come back. highly distributed, no form of blocking would be effective except maybe captcha or proof of work.

lucb1e 5mo ago

Then block all of Amazon, Meta, or wherever botnet/crawling traffic is coming from that doesn't honor robots.txt, sends DDoS reflection traffic, submits SMTP messages (in large volumes, not just probing) for domains they're not authorized for with SPF, or whatever else applies to the protocol you're using If they can't keep their ranges clean to a reasonable degree, their customers will need to move if they want to access your part of the internet. New sign-ups will always be hard, so some amount of abuse is expected, but if it's the same abuse traffic for weeks after you've notified them, well, it stops being your problem at some point

Jolter 5mo ago
See the other comments in this thread. The perpetrators are unknown and are jumping between residential IPs. Possibly botnets?
- lucb1e 5mo ago
  Then see my other replies in the thread where I've specifically addressed residential IPs, e.g.: https://news.ycombinator.com/item?id=48163060 https://news.ycombinator.com/item?id=48163060
  - Jolter 5mo ago
    This is the post I’m talking about. Make sure you understand how it would not be productive to go after each ISP individually when the traffic is from all of them. https://news.ycombinator.com/item?id=48155512 https://news.ycombinator.com/item?id=48155512