3 ms·
Why should I use this instead of just firing up some spot instances with phantomjs?
by Eridrus 11y ago
Why should I use this instead of just firing up some spot instances with phantomjs?
- jancurn 11y agoYou could definitely do that, but then you might also need to: - implement a mechanism to find and click active page elements, track the browser actions - recompile PhantomJS to support POST requests - implement a page queue with checks for duplicate URLs - implement some parallelization and failover mechanism for case PhantomJS crashes (it does) - possibly implement support for infinite scroll - setup your own pool of proxy servers - setup a database to store the results and finally make the whole thing simple to setup and use :)