3 ms·
This is really interesting. One naive thought that immediately came to mind is that bots might be capable of making cross site requests. The logical conclusion
by apitman 1y ago
This is really interesting. One naive thought that immediately came to mind is that bots might be capable of making cross site requests. The logical conclusion of this entire arms race is that bots will eventually have no choice but to run actual browsers. Not sure that fact will appreciably reduce their scaping abilities though.
- benregenspan 1y ago> The logical conclusion of this entire arms race is that bots will eventually have no choice but to run actual browsers I think this is almost already the case now. Services like Cloudflare do a pretty good job of classifying primitive bots and if site operators want to block all (or at least vast majority), they can. The only reliable way through is a real browser. (Which does put a floor on resource needs for scraping)
- dragonwriter 1y ago> The logical conclusion of this entire arms race is that bots will eventually have no choice but to run actual browsers. I thought bots using (headless) browsers was an existing workaround for a number of existing issues with simpler bots, so this doesn't seem to be a big change.