3 ms·
Cloud solutions have plenty of issues, but I'm quite surprised there aren't more replies talking about how many impossible problems have been made tractable and
by stefco_ 7y ago
Cloud solutions have plenty of issues, but I'm quite surprised there aren't more replies talking about how many impossible problems have been made tractable and reliable thanks to the cloud (or to be more precise, made much cheaper to solve reliably). The article makes a great point about accountability, in the sense that no one at a cloud-providing corp is immensely worried about transient failures that only affect small user sets heavily (which sucks if you're affected). But for my scientific computing use cases, getting things working reliably at any sort of scale within budget is impossible without the cloud. Research institutions' computing clusters are just smaller, less reliable, less flexible versions of the cloud (good luck getting sysadmins to do anything useful at all).
One of the collaborations I work in, LIGO, recently gave up on private servers and transitioned to AWS for our Gravitational Candidate Database [1] because the cloud is so much better. I made this change to my own low-latency search framework [2] years ago. If you're not "lucky" enough to (be forced to) use a university/collaboration cluster, you'd have to maintain your own server, which is orders of magnitude less reliable and more expensive/difficult. I understand that not all workflows are the same, but for all of my nontrivial applications, cloud providers save so much time and money that I can do something as bold as making a provider-agnostic architecture with more robust failover. I recognize that more complicated workflows might require e.g. 10 separate AWS services with AWS-specific features causing lock-in, but at that level of complexity, I'm guessing the problem must be virtually impossible with a non-cloud solution anyway. If you really can't figure out another way to deal with resiliency, you might just need to accept that your problem space is really hard and that you're lucky to even be able to run it at all. Again, I think the original article is right about the fact that you have to account for this yourself; the cloud is not magic, and your code still has to understand that it is (like all abstractions) going to leak.
Again, the point about responsiveness in the original article is very well-taken; I'm just surprised more people aren't observing that overall the reliability, cost, and flexibility provided by cloud solutions is utterly transformative in terms of reliability.
[1] http://gracedb.ligo.org http://gracedb.ligo.org
[2] http://multimessenger.science http://multimessenger.science