3 ms·
Not a user of this tool, but https://github.com/yujiosaka/headless-chrome-crawler#event-newpage https://github.com/yujiosaka/headless-chrome-crawler#event-n...
by diggan 9y ago
Not a user of this tool, but https://github.com/yujiosaka/headless-chrome-crawler#event-newpage https://github.com/yujiosaka/headless-chrome-crawler#event-n... points to https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-page https://github.com/GoogleChrome/puppeteer/blob/master/docs/a... where you can grab elements.
Basically, when new page event happens, you get the `page` object where you have access to it and can do queries.
- ej12n 9y ago`page.$(query)` or `page.$$(queryAll)` to be more specific https://github.com/GoogleChrome/puppeteer/blob/v1.1.0/docs/api.md#pageselector https://github.com/GoogleChrome/puppeteer/blob/v1.1.0/docs/a...
- bryanrasmussen 9y agoyes I've used puppeteer, but I couldn't see where it exposed the page object, but looking closer I saw HCCrawler.launch({ // Function to be evaluated in browsers evaluatePage: (() => ({ title: $('title').text(), })), // Function to be called with evaluated results from browsers onSuccess: (result => { console.log(result); }), }) which doesn't look that good for my other worry, that you come to a dynamic JS built up page, load the DOM and evaluate in some milliseconds and then that DOM is changed after you've done your incorrect evaluation.