I indirectly worked on a project for a company trying to create an in house web indexer to track user generated content to find trends. Like 80% of the work wa

I indirectly worked on a project for a company trying to create an in house web indexer to track user generated content to find trends. Like 80% of the work was making it look like the web scrapers weren't coming from an IP in the cloud. The IPs for the different cloud providers are super well known, and the big guys either put you behind captcha hell, or just flat out block you if you're coming from one of them.