As someone who may or may not monitor very elastic competitor pricing (allegedly) I would add that if you are not respectful with your scraping you will get IP

As someone who may or may not monitor very elastic competitor pricing (allegedly) I would add that if you are not respectful with your scraping you will get IP banned. You might also get shut down by a provider (e.g. aws). Services like Crawlera act as a proxy that also forces you to be nice. If anyone is getting into this, I would recommend they use a service like that to scrape at scale.