depends. if you also do tracking for valid navigation paths then rate limiting may be effective. For instance: if you have a search result with a 1000 pages th

depends. if you also do tracking for valid navigation paths then rate limiting may be effective. For instance: if you have a search result with a 1000 pages that someone is trying to scrape if you don't allow people to jump into the middle of the result set then just rotating IPs doesn't work.