I use scrapy, although it doesn't do great w/ JS heavy sites unless you can figure out the ajax request and just simulate those. Tutorials are pretty easy to f

I use scrapy, although it doesn't do great w/ JS heavy sites unless you can figure out the ajax request and just simulate those. Tutorials are pretty easy to find, as it's the top python based scraping tool.