4 ms·
This is not a good way to do this. Tor exit nodes are public and may be marked for special behavior by the marketplace you are surveying. There is no reason to
by cakealert 1y ago
This is not a good way to do this. Tor exit nodes are public and may be marked for special behavior by the marketplace you are surveying. There is no reason to believe you are getting good information this way.
The right way to do this would be through a VPN/tor + Residential proxy to hide your intentions from everyone involved.
- neilv 1y ago> There is no reason to believe you are getting good information this way. Spot checks checked out. And it was a perfectly fine way to do it. You are correct that Tor exit nodes often get special handling (at the moment, including by Cloudflare, and by Google Recaptcha). And the idea of poisoning of data is starting to propagate, due do anti-AI-scraper sentiment.
- tomschlick 1y agoPoisoning the data has been around for at least a decade now. I used to work for a very large pricing analytics company that would track product pricing for Fortune 500 manufacturers. We found on several occasions that some shady retailers would find the CIDR of the manufacturers corp networks and comply with the MAP policies on pricing if traffic came from them. Then when our bot went through with obviously generic AWS / Proxy ips we would see a much lower price that broke their agreement. That one was a fun realization for the manufacturers as to the level of shadiness some retailers would go through.