Disclaimer: I don't have as much experience with phantomjs and the many more traditional services mentioned due to sticking mainly to jvm languages for my backe

Disclaimer: I don't have as much experience with phantomjs and the many more traditional services mentioned due to sticking mainly to jvm languages for my backends. That being said: I've run in to a lot of problems scraping sites with heavy javascript. I personally use selenium with a headless google chrome for it. http://www.alittlemadness.com/2008/03/05/running-selenium-headless/ http://www.alittlemadness.com/2008/03/05/running-selenium-he... Granted, there's a bit of custom logic involved at times, but it's by far the most reliable compared to the alternatives I've seen like HTMLUnit. It's allowed me to just focus on worrying about page content rather than worrying about if something will work or not.