A missing elements for me is that with a lot of exploratory scientific work we (often half intentionally) have no idea what we are doing. We can easily run a gi

A missing elements for me is that with a lot of exploratory scientific work we (often half intentionally) have no idea what we are doing. We can easily run a giant job that uses 100x more compute than expected. Yes you can limit cloud compute resources if you are smart but its much better is the default is that you run out of compute and your job takes longer than you get a 100x bill back from our cloud provider. And if you limited your cloud resources to a fixed amount, didn't you just eliminate half the benefit of using cloud in the first place? Then the problems of data management, transfer and egress are huge. Again the "no idea what we are doing" factor comes into play. If you have a really good idea up front what is going to happen you can plan out a strategy that minimises costs. But if you have no idea at all - genuinely, because this is science and we are doing new things - then you could end up blowing huge amounts of money on unnecessary egress and storage costs. And at the small end we can be talking about experiments run on a shoestring where a few thousand dollars is a big deal. The way I see it, we need everything - powerful individual workstations / laptops for direct analysis, then a tier of fixed HPC style compute for this kind of work that is poorly matched to cloud, and then for specific projects where it makes sense (massive scaleout, exotic hardware needs - GPUs, FPGAs etc) you embrace cloud resources for that.