They say this: > We have IP-based rate limiting in place for many of our endpoints, however these crawlers are coming from a large number of IP addresses, so o

They say this: > We have IP-based rate limiting in place for many of our endpoints, however these crawlers are coming from a large number of IP addresses, so our rate limiting is not effective. Do you have something else in mind? Just shut down the whole site after a certain limit?