4 ms·
Same as always - proxy farms, random popular UAs with random delays etc.
by hdjjhhvvhga 5y ago
Same as always - proxy farms, random popular UAs with random delays etc.
- juanse 5y agoSorry, what is a UA?
- bryanrasmussen 5y agoso will Google's freezing of the UA lead to less ability to web scrape for the non big company scrapers out there?
- vivekv 5y agoSorry can you please elaborate what is Google doing?
- bryanrasmussen 5y agosorry, I thought it was a well known thing here given the various discussions over past year or so https://groups.google.com/a/chromium.org/g/blink-dev/c/-2JIRNMWJ7s/m/yHe4tQNLCgAJ https://groups.google.com/a/chromium.org/g/blink-dev/c/-2JIR... on edit: so I'm thinking as there will only be one UA floating around then, sure, older UAs can exist, but those become progressively more suspicious.
- hdfinenwvdun 5y agoYou can easily spoof the UA
- dreyfan 5y agoWhat? It’s just a text string in the header. How in the world would that possibly make it more difficult to scrape? All Chrome is doing is stop appending the current semver in the UA it sends.
- bryanrasmussen 5y agoI mean I realize it probably isn't a problematic, just wondering, but on the other hand it shouldn't be so difficult to follow the reasoning based on the context I would think: poster says - in order to be able to scrape effectively you should appear to be a real human, use different UAs etc. So as this change happens different UAs become one less thing that you can easily change to seem less suspicious, as a non-frozen UA would then be a suspicious sign after some time. So a sort of side effect.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- heavyset_go 5y agoThe switch from UA to browser fingerprinting makes it harder to scrape without being stopped. Yes, at any time the UA could be ignored and clients could be fingerprinted, but now the UA is being made next to useless, so fingerprinting will now become the default everywhere.