3 ms·
I had the same thing happen out of the azure data center in Des Moines Iowa. I ended up having to geolocate the IP address in combination with a few funky / mis
by patja 4y ago
I had the same thing happen out of the azure data center in Des Moines Iowa. I ended up having to geolocate the IP address in combination with a few funky / missing request headers to fingerprint this miscreant. They were rotating through hundreds of IP addresses.
It was a really dumb spider. It wasted a huge amount of its efforts requesting pages that are no longer valid on my site. It must have had an archived set of URLs it was using as a source rather than spidering the site as it is today.