We at Rewritely (http://www.rewritely.com http://www.rewritely.com) have quite some experience in dealing large scale content migration for clients (not just a

We at Rewritely (http://www.rewritely.com http://www.rewritely.com) have quite some experience in dealing large scale content migration for clients (not just a single HTML page, but a whole site) mainly using scraping techniques. We have seen so many invalid, funny, ugly markups and edge cases and eventually, we found out the only reliable parser is actually the "browser" you use everyday - which is heavily tested by hundreds of millions of users! We chose to run the scraper in a real browser so what you scrape is what you see, not to mention you can fire up events, Ajax requests, inject JavaScript or even play Flash video... they are possible only when you have a real browser. Also, the scraper codes need to be concise and expressive (keys: maintainable & testable), because your codes are going to break sooner or later - if you are doing serious scraping business. Less LOC = more easy to change. JavaScript & jQuery are the obvious winners in this category and what is more, they are fun to work with. Disclaimer: we have no relationship with PhantomJS :)