Looks good thanks for sharing, you could also use an anonymous scraping API to scale your scraper up if you are crawling on high volumes for instance using this

Looks good thanks for sharing, you could also use an anonymous scraping API to scale your scraper up if you are crawling on high volumes for instance using this node package. https://www.npmjs.com/package/proxycrawl https://www.npmjs.com/package/proxycrawl