3 ms·
What does headless Chrome provide that a web scraper in any given language can't?
by Dirlewanger 9y ago
What does headless Chrome provide that a web scraper in any given language can't?
- dragonwriter 9y ago> What does headless Chrome provide that a web scraper in any given language can't? Everything a web scraper could do, without reinventing all the infrastructure for handling web content (including JS/DOM interaction) from scratch. You could obviously do it all yourself in your language of choice,but why not focus on the application specific parts?
- wingerlang 9y agoI was hoping it would let me automate a few tasks with a "user" flow (i.e. enter details, click, click). With e.g. curl or python I didn't even get through the login screens because they seem to require some special dealing with cookies, request/response cookies, smfd, itc, id_ado, _ip_xat and so on. Basically auth on websites seems to require a whole bunch of stuff and I just started looking at simpler forms of automation instead. Still not sure which one I'll continue looking into. (If anyone sees this, I am trying to log into iTunes Connect and Fabric and download metrics)
- VeejayRampay 9y agoTaking screenshots comes to mind. It's still extremely complicated to this day to get it right. Even with the likes of PhantomJS, Chrome Headless, etc.