2 ms·
If you know which requests you need to do in order to get the info you want instead of waiting for the headless browser to execute the javascript code that rend
by iamsvera 10y ago
If you know which requests you need to do in order to get the info you want instead of waiting for the headless browser to execute the javascript code that renders the needed data, it is pretty simple, you just need an http client and something to parse the content, depending if is html I'd recommend using cheerio in NodeJS to parse and traverse the content with a jQuery like API.
If you must scrape the content with a headless browser, I'd choose Selenium.
https://github.com/sourcegraph/go-selenium https://github.com/sourcegraph/go-selenium
- iamsvera 10y agoThere is a really cool and easy to use library for NodeJS, but it doesn't use a headless browser :( it just makes the requests and parses the content. https://github.com/IonicaBizau/scrape-it https://github.com/IonicaBizau/scrape-it
- tmaly 10y agoThanks, have you used go-selenium and what did it take to get up and going with it?