4 ms·

I see both sides of the argument, there is a reason why CERN is not processing their data using EC2 and lambdas.

harunurhan 4y ago
The cost isn't the only reason - CERN started planning its computing grid before AWS was launched. - It's pretty complicated (politics, mission, vision) for CERN to use external proprietary software/hardware for its main functions (they have even started to MS Office like products.) - [cost] CERN is quite different than a small team researchers doing few years research. the scale is enormous and very long lived, like for decades continue - and more... HPC and scientific computing aside, I would have loved to be able to use AWS when I worked there, internal infra for running web apps and services wasn't nearly good & reliable, neither had a wide catalog of services offered.
- betolink 4y ago
  I think the spirit of the article is to put the cloud in perspective of the organization size and the workload type. There is a sweet spot where the cloud is the only option that makes sense, definitely with variable loads and capacity to basically scale on demand as big as our budget, there is no match for that. However... there are organizations with certain type of workloads that could afford to put infrastructure in place and even with the costs of staffing, energy etc they will save millions in the long run. NASA, CERN etc are some. This is not limited to HPC, the cloud at scale is not cheap either see: https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap-cloud-lifecycle-scale-growth-repatriation-optimization/ https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...
thamer 4y ago
The vast majority of researchers don't need anywhere close to the amount of resources that CERN needs. The fact that CERN doesn't use EC2 and lambdas shouldn't be taken as a lesson by anyone who's not operating at their scale. This feels like a similar argument to the one made by people who use Kubernetes to ensure their web app with 100 visitors a day is web scale.

The cost isn't the only reason - CERN started planning its computing grid before AWS was launched. - It's pretty complicated (politics, mission, vision) for CERN to use external proprietary software/hardware for its main functions (they have even started to MS Office like products.) - [cost] CERN is quite different than a small team researchers doing few years research. the scale is enormous and very long lived, like for decades continue - and more... HPC and scientific computing aside, I would have loved to be able to use AWS when I worked there, internal infra for running web apps and services wasn't nearly good & reliable, neither had a wide catalog of services offered.

betolink 4y ago
I think the spirit of the article is to put the cloud in perspective of the organization size and the workload type. There is a sweet spot where the cloud is the only option that makes sense, definitely with variable loads and capacity to basically scale on demand as big as our budget, there is no match for that. However... there are organizations with certain type of workloads that could afford to put infrastructure in place and even with the costs of staffing, energy etc they will save millions in the long run. NASA, CERN etc are some. This is not limited to HPC, the cloud at scale is not cheap either see: https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap-cloud-lifecycle-scale-growth-repatriation-optimization/ https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...

thamer 4y ago

The vast majority of researchers don't need anywhere close to the amount of resources that CERN needs. The fact that CERN doesn't use EC2 and lambdas shouldn't be taken as a lesson by anyone who's not operating at their scale. This feels like a similar argument to the one made by people who use Kubernetes to ensure their web app with 100 visitors a day is web scale.