I have administered an HPC cluster for a research group working on genetics and genomics. There was some data that was the “secret sauce” mostly because it was

I have administered an HPC cluster for a research group working on genetics and genomics. There was some data that was the “secret sauce” mostly because it was curated over a long period of time with industrial collaborators and bound by several agreements that it would be a huge task in itself to navigate the proper permissions to ever release them publicly. The default policy was to consider all of them secret and available only on a project need basis for researchers closely associated with the institute. Now, it is technically possible to have this data in AWS with assurances of encryption and some sane security measures like not leaving it open to public. However, the big catch is - the compute part of the cluster is on-premise. We did the math of moving the compute to AWS, and it was several orders of magnitude expensive(consider that the group did not need to pay for power, network links, and maintenance of the server room by virtue of being part of the university). The only “extra cost” of being on-premise was my salary, which is substantially small portion of the increase by going to AWS. Also, guess what going to AWS doesn’t magically eliminate the need for a guy whose responsibility it is to ensure the cluster is operated, and for users to come in for help, and to manage the ecosystem in general. The argument of moving to the cloud is more complicated than it appears on the surface. I believe the HPC shops that have a lot of compute and very little need for elasticity will be the last to head to the cloud.