3 ms·
Have you verified that they are actually Amazon crawlers as outlined here: https://developer.amazon.com/amazonbot https://developer.amazon.com/amazonbot
by alphan0n 2y ago
Have you verified that they are actually Amazon crawlers as outlined here:
https://developer.amazon.com/amazonbot https://developer.amazon.com/amazonbot
- xena 2y agoYes.
- alphan0n 2y agoCan you share the list of offending ip’s, here or your website so we can use them in block lists? Also, there is an email to contact: amazonbot@amazon.com
- xena 2y agoThe website that you were pummeling is in the article: git.xeserv.us. I sent an email earlier today and have gotten no response. Right now your crawler bots are getting the bee movie script, so you may want to delete all the data that's being scraped from that domain. Unless you like jazz that is. It'd be a gesture of good faith to remunerate me for the egress fees your bot incurred, but I'm not gonna die on that hill.
- alphan0n 2y agoApologies, I’m not affiliated with Amazon in any way. I meant the Amazon ip addresses that are causing you trouble so I can preemptively block them.