4 ms·
My last web scraping activity consisted of using PhantomJS to drive and injecting my own javascript that submitted the pertinent data to my own web service. ht
by epoxyhockey 15y ago
My last web scraping activity consisted of using PhantomJS to drive and injecting my own javascript that submitted the pertinent data to my own web service. http://phantomjs.org/ http://phantomjs.org/
- mistercow 15y agoWow, I had used node+jsdom+jQuery to do scraping, which was a big step up from anything I had done previously. But after playing around with PhantomJS, I feel like I finally found the hammer in the toolbox after years of trying to pound nails in with duct-taped-together screwdrivers.
- epoxyhockey 15y agoCheck that.. I wanted to use js to submit to my own web service, but forgot about the cross-domain issues, so I just used console.log to output the data. I suppose I could have used the iframe querystring parameter trick.
- easp 15y agoHmmm, seems like something you could have worked around by running phantom.js against a proxy that intercepted and rerouted requests to the data collection server.
- zackzackzack 15y agoIt's hella slow but there is always this: https://github.com/sgentle/phantomjs-node https://github.com/sgentle/phantomjs-node
- uptown 15y ago"I suppose I could have used the iframe querystring parameter trick." How's that work? Pass whatever data you want to process with your web service as URL parameters to the web service src URL of your iframe and it circumvents cross-site-scripting protections?
- deleted 15y ago[deleted]