3 ms·
I used headless selenium successfully for executing javascript. it has a nice feature of waiting the browser until the some element appears in the HTML.
by level09 13y ago
I used headless selenium successfully for executing javascript. it has a nice feature of waiting the browser until the some element appears in the HTML.
- al1x 13y agoI found all of the available JS engines in Selenium to be insufficient for my purposes. None of them handled a wide range of javascript well and would often fail to execute it. I switched to PhantomJS and couldn't be happier.
- 91bananas 13y agoHere is another little snippet I used to scrape yelp before. It involved polling for a specific element on the page, if you can inject or find jQuery, this is pretty light and simple. edit: Should note that jQuery is less than necessary for this (since I'm sure someone would have said it right after I posted this), just makes me have to type less crap :) https://gist.github.com/91bananas/5644737 https://gist.github.com/91bananas/5644737