3 ms·
I'm looking for something like this but for scraping SPAs and JS-rich web content. A single static binary like this not chrome driver or selenium etc
by ducktective 4y ago
I'm looking for something like this but for scraping SPAs and JS-rich web content. A single static binary like this not chrome driver or selenium etc
- mplewis9z 4y agoGood luck with that - you'd need to have a fully standards-compliant HTML/CSS/JS engine in order to get the same output, and at that point that's literally just a browser (and would require a massive investment of time for maintenance). A driver for an existing browser is the only reasonable option for the foreseeable future, with how complex modern web development has become.
- chrismarlow9 4y agoYou need something to replace the curl part of this, not pup. You can pipe js into chrome via CLI with --repl flag and dump the full Dom and then pipe that into pup. You'll be stuck with GET requests only though, and very simple ones, unless you get creative with the js you pipe in.