We have 500-node cluster at a chemical company, and we've been experimenting with "hybrid-cloud". This allows jobs to use servers with resources we just don't h

We have 500-node cluster at a chemical company, and we've been experimenting with "hybrid-cloud". This allows jobs to use servers with resources we just don't have, or couldn't add fast enough. Storage is a huge issue for us. We have a petabyte of local storage from big name vendor that's bursting at the seams, and expensive to upgrade. A lot of our users leave big files laying around for a large time. Every few months we have to hound everyone to delete old stuff. The other thing that you get with the cloud is there's way more accountability for who's using how much resources. Right now we just let people have access and roam free. Cloud HPC is 5-10x more in cost and the beancounters would shut shit down real quick if the actual costs were divvied up. We also still have a legacy datacenter so in a similar vein, it's hard to say how much not having to deal with physical hardware/networking/power/bandwidth would be worth. Our work is maybe 1% of what that team does.