The absolute best tool i have found for scraping is Visual Web Ripper. It is not open source, and runs in windows only, but it is one of the easiest to use too

The absolute best tool i have found for scraping is Visual Web Ripper. It is not open source, and runs in windows only, but it is one of the easiest to use tools that i have found. I can set up scrapes entirely visually, and it handles complex cases like infinite scroll pages, highly javascript dependent pages and the like. I really wish there were an open source solution that was as good as this one. I use it with one of my clients professionally. Their support is VERY good btw. http://visualwebripper.com/ http://visualwebripper.com/