4 ms·

You missed what I think is the most important point in OP's list: it does all of the above in a cloud agnostic way. If I want to move clouds with TF I'm rewriti

You missed what I think is the most important point in OP's list: it does all of the above in a cloud agnostic way. If I want to move clouds with TF I'm rewriting everything to fit into a new cloud's paradigm. With Kubernetes there's a dozen providers built in (storage, loadbalancing, networking, auto scaling, etc.) or easy to pull in (certificates, KMS secrets, DNS); and they make moving clouds (and more importantly) running locally much easier. Kubernetes is currently the best way to wrap up workloads in a cloud agnostic way. I've written dozens of services for K8s using different deployment mechanisms (Helm, Carvel's kapp, Flux, Kustomize) and I can run them just as easily in my home K8s cluster and in GCP. It's honestly incredible; I don't know of any other cloud tech that lets me do that. One thing I think a lot of people miss too, is how good the concepts around Operators in Kubernetes are. It's hard to see unless you've written some yourself, but the theory around how operators work is very reminiscent of reactive coding in front end frameworks (or robotics closed loop control, what they were originally inspired by). When written well they're extremely resilient and incredibly powerful, and a lot of that power comes from etcd and the established patterns they're written with. I think Kubernetes is really painful sometimes, and huge parts of it aren't great due to limitations of the language it's written in; but I also think it's the best thing available that I can run locally and in a cloud with a FOSS license.

dijit 3y ago
> it does all of the above in a cloud agnostic way. I'll give you the benefit of the doubt here and say that some of the basics are indeed cloud agnostic. However, it's plain for many or most to see that outside of extremely "toy" workloads you will be learning a specific "flavour" of Kubernetes. EKS/GKE/AKS etc; They have, at minimum, custom resource definitions to handle a lot of things and at their worst have implementation specific (hidden) details between equivalent things (persistent volume claims on AWS vs GCP for example is quite substantially different).
- theossuary 3y ago
  For multicloud I usually think of my local K8s cluster and GKE, it's been a few years since I touched EKS. I'd love to hear your opinions on the substantive differences you run into. When switching between clouds I'm usually able to get away with only changing annotations on resources, which is easy enough to put in a values.yml file. I can't remember the last time I had to use a cloud specific CRD. What CRD's do you have to reach for commonly? Thinking about it; the things I see as very cloud agnostic: Horizontal pod autoscaling, Node autoscaling, Layer 4 loadbalancing, Persistent volumes, Volume snapshots, Certificate managment, External DNS, External secrets, Ingress (when run in cluster, not through a cloud service), That ends up covering a huge swath of my usecases, probably 80-90%. The main pain points I usually run into are: IAM, Trying to use cloud layer 7 ingress (app loadbalancers?) I totally agree the underlying implementation if resources can be very different, but that's not the fault of Kubernetes; it's an issue with the implementation from the operator of the K8s cluster. All abstractions are going to be leaky at this level. But for PVCs I feel like storageclasses capture that well, and can be used to pick the level of performance you need per cloud; without having to rewrite the common provision of block device.
elktown 3y ago
Something feels very off and mantra-like with the proportionality of how often cloud migration benefits are being presented as something very important to how often that actually happens in practice. Not to even mention that it also assumes that simpler setups are automatically harder to move around between clouds, or at least that there are a significant difference in required effort.
- theossuary 3y ago
  When I say it's easy to move between clouds, I'm not referring to an org needing to pick up everything and move from AWS to GCP. That is rare, and takes quite a bit of rearchitecting no matter what. When I say something is easy to move, I mean that when I build on top of it, it's easy for users to run it in their cloud of choice with changes in config. It also means I have flexibility with where I choose to run something after I've developed it. For example I develop most stuff against minikube, then deploy it to GCP or a local production k8s. If I was using Terraform I couldn't do that.
  - deleted 3y ago
    [deleted]
  - elktown 3y ago
    Not sure what kind of apps this is but I can't see the big value-add on a golang app binary wrt to being cloud agnostic, nor wrt local development. It makes even less assumptions on the user's env. Still need some cloud conf (terraform, database etc) either way If you'll excuse a slight digression, but I think there's a tendency atm to rather pay $1 in extra complexity a 100 times over time, than pay a $5 one-time fee. Like if repeating something similar twice - even if it's easy and not really a lot of effort - is a sign of failure and thus unbearable.

dijit 3y ago

> it does all of the above in a cloud agnostic way. I'll give you the benefit of the doubt here and say that some of the basics are indeed cloud agnostic. However, it's plain for many or most to see that outside of extremely "toy" workloads you will be learning a specific "flavour" of Kubernetes. EKS/GKE/AKS etc; They have, at minimum, custom resource definitions to handle a lot of things and at their worst have implementation specific (hidden) details between equivalent things (persistent volume claims on AWS vs GCP for example is quite substantially different).

theossuary 3y ago
For multicloud I usually think of my local K8s cluster and GKE, it's been a few years since I touched EKS. I'd love to hear your opinions on the substantive differences you run into. When switching between clouds I'm usually able to get away with only changing annotations on resources, which is easy enough to put in a values.yml file. I can't remember the last time I had to use a cloud specific CRD. What CRD's do you have to reach for commonly? Thinking about it; the things I see as very cloud agnostic: Horizontal pod autoscaling, Node autoscaling, Layer 4 loadbalancing, Persistent volumes, Volume snapshots, Certificate managment, External DNS, External secrets, Ingress (when run in cluster, not through a cloud service), That ends up covering a huge swath of my usecases, probably 80-90%. The main pain points I usually run into are: IAM, Trying to use cloud layer 7 ingress (app loadbalancers?) I totally agree the underlying implementation if resources can be very different, but that's not the fault of Kubernetes; it's an issue with the implementation from the operator of the K8s cluster. All abstractions are going to be leaky at this level. But for PVCs I feel like storageclasses capture that well, and can be used to pick the level of performance you need per cloud; without having to rewrite the common provision of block device.

elktown 3y ago

Something feels very off and mantra-like with the proportionality of how often cloud migration benefits are being presented as something very important to how often that actually happens in practice. Not to even mention that it also assumes that simpler setups are automatically harder to move around between clouds, or at least that there are a significant difference in required effort.

theossuary 3y ago
When I say it's easy to move between clouds, I'm not referring to an org needing to pick up everything and move from AWS to GCP. That is rare, and takes quite a bit of rearchitecting no matter what. When I say something is easy to move, I mean that when I build on top of it, it's easy for users to run it in their cloud of choice with changes in config. It also means I have flexibility with where I choose to run something after I've developed it. For example I develop most stuff against minikube, then deploy it to GCP or a local production k8s. If I was using Terraform I couldn't do that.
- deleted 3y ago
  [deleted]
- elktown 3y ago
  Not sure what kind of apps this is but I can't see the big value-add on a golang app binary wrt to being cloud agnostic, nor wrt local development. It makes even less assumptions on the user's env. Still need some cloud conf (terraform, database etc) either way If you'll excuse a slight digression, but I think there's a tendency atm to rather pay $1 in extra complexity a 100 times over time, than pay a $5 one-time fee. Like if repeating something similar twice - even if it's easy and not really a lot of effort - is a sign of failure and thus unbearable.