From a data analytics perspective, it is surprising to me if I don't find multiple opportunities for order of magnitude improvements in data processing infrastr

From a data analytics perspective, it is surprising to me if I don't find multiple opportunities for order of magnitude improvements in data processing infrastructure at my clients. At one client where I was doing more IC dev work, it got to the point where they'd just chuck reports over the wall at me and ask me to find efficiencies in these. This was supporting >15K internal users in a massive shared reporting and BI infrastructure, so it was well worth it to have someone spending 10s of hours on optimization on a regular basis. I could routinely find order of magnitude improvements in reporting queries that were already running in production. If bringing things from POC/end user mockup to production, I could frequently find two orders of magnitude of improvement in query time, which reflected directly in latency to report rendering for end users. That client was not special. I wouldn't call this sort of work casual, and it's a very different domain than optimizing serving web apps. I would spend days to weeks on such optimization tasks at the client I mentioned above. It might take a few weeks to do serious optimization work on large ETL pipelines, but I routinely find big gains in performance on such projects - usually in the range of 1/2-1/10 run time compared to baseline.