Haven't tried this[0] yet, but Scrapy should be able to handle JavaScript sites with the JavaScript rendering service Splash[1]. scrapy-splash[2] is the plugin

Haven't tried this[0] yet, but Scrapy should be able to handle JavaScript sites with the JavaScript rendering service Splash[1]. scrapy-splash[2] is the plugin to integrate Scrapy and Splash. [0] https://blog.scrapinghub.com/2015/03/02/handling-javascript-in-scrapy-with-splash/ https://blog.scrapinghub.com/2015/03/02/handling-javascript-... [1] https://splash.readthedocs.io/en/stable/index.html https://splash.readthedocs.io/en/stable/index.html [2] https://github.com/scrapy-plugins/scrapy-splash https://github.com/scrapy-plugins/scrapy-splash