4 ms·
There's a ton at play here. "Managed Kubernetes" really runs the spectrum between "one step above just installing it yourself on a bunch of VMs" and "I spend 1
by 013a 7y ago
There's a ton at play here.
"Managed Kubernetes" really runs the spectrum between "one step above just installing it yourself on a bunch of VMs" and "I spend 1% of my time managing anything below the product." Each cloud provider exists somewhere different on this spectrum, with none of them being in quite the same location, and some of them have multiple different products which exist at different points.
For example: AWS is among the most bare-bones. EKS is just a managed control plane; coming from GKE, you might click "create an cluster" then be very confused how there are no options for, say, instance size, or how many... because you have to do that all yourself. There are tools like eksctl or Rancher which can help with this, but ultimately, you're managing those instances. You're doing capacity planning (you think kube would be a great pick to integrate with spot fleets because of its ability to schedule and move workloads to a new instance when one goes down? have fun setting it up, hope you like ops work.). You're doing auto-scaling (and that ASG? its not going to know about your pod resource requests, so you either need some very smart manual coordination between the two, or you need to set up cluster-autoscaler). You're setting up cluster metrics (definitely need metrics-server. not heapster, that was last year, metrics-server is this year. but how to visualize? do i host grafana in the cluster? then i need to worry about authn. cloudwatch really isn't made for these kinds of things... maybe I'll just give datadog a few thousand bucks.) Crap, 1.16 is out already? They only support 9 months of releases with security updates?! I feel like I just upgraded my nodes! Oh well, time to lose a day replicating this update across all of my environments.
I'd go on, but you get the point. There is nothing "managed" about EKS.
DigitalOcean is pretty similar to this (it does provision instances, but the tooling beyond that is barebones). Google Cloud/GKE is "more managed" in a few sense; the cloud dashboard provides some great management capabilities out-of-the-box, such that you may not need to reach for something like Datadog, and the autoscaler works really well without a lot of tinkering. There are still underlying instances, so you're worrying about ingress protection, OS hardening, OS upgrades, etc... but its not as bad as AWS. Not by a long shot.
The holy grail (for some companies) is really something like Azure AKS + Azure Container Instances. No instances to manage. Click a button for a kubernetes cluster. Schedule workloads. Get functional metrics, logging, tracing, dashboards out of the box. Don't worry about OS upgrades, hardening, autoscaling, upgrading the cluster, etc; we'll do it all for you, or at least make it one click to configure. That's the ideal situation. I haven't used AKS/ACI so I can't comment on whether Azure gets us there, but the idea is sound; even if its more expensive.
This sounds like an anti-Kube post, right? Wrong (its a Tide ad). The beautiful thing about Kubernetes is that it can span this spectrum. The same exact API surface can scale from a fully-managed abstract platform where you just say "take this git repo and run it" (see: Gitlab Auto-DevOps), all the way to powering millions of workloads across dozens of federated clusters at Fortune 500 companies.
But, to the OPs point: We're close to solving that right end of the spectrum, and a lot further away from the left end. We're getting there, but we're not there yet. There isn't enough abstracted management of these compute resources... yet. But there's enough money and desire for there to be that I know we'll get there.
- manigandham 7y agoGKE and AKS both require setting up instance types for node pool(s) but you don't need to worry about the OS and related issues on either. Both also support auto-scaling and auto-upgrades. GKE is far more managed though with advanced features like global network, aliased IPs, global load-balancing, istio/traffic manager integration, private IPs, metrics-based-autoscaling, preemptible nodes, local SSDs, GPUs, TPUs, etc. Azure Container Instances are nice, with the GCP corollary being Cloud Run using KNative. Both products are designed to quickly run a container image with a public endpoint, but ACI is not part of AKS. You might be thinking of the Azure Virtual Kubelet which lets you burst on-demand to ACI, but this is a very advanced use-case and not the normal cluster setup.
- deepsun 7y ago> The holy grail (for some companies) is really something like Azure AKS + Azure Container Instances. No instances to manage... Google Cloud have been having that since before it's being called Google Cloud -- I mean Google App Engine. It has all those features, and with its "Standard" environment you don't even need to build docker image.
- 9nGQluzmnq3M 7y agoCloud Run is even closer. App Engine Standard requires you to use the sandbox, App Engine Flex is tied to VMs, but Cloud Run makes containers serverless: https://cloud.google.com/run/ https://cloud.google.com/run/ ...and since it's based on Knative, it's portable-ish.
- gorbypark 7y agoI can attest that Cloud Run has been great for me. I literally "ported" over some of my apps that were already running in containers by replacing references to my existing env PORT_NUM to GCR's PORT and it just worked. It's been a life changer. Now if it could run non-http workloads, I'd be able to shut down every one of my VMs.
- bproven 7y ago>For example: AWS is among the most bare-bones. EKS is just a managed control plane; coming from GKE, you might click "create an cluster" then be very confused how there are no options for, say, instance size, or how many... because you have to do that all yourself. There are tools like eksctl or Rancher which can help with this, but ultimately, you're managing those instances. You're doing capacity planning (you think kube would be a great pick to integrate with spot fleets because of its ability to schedule and move workloads to a new instance when one goes down? have fun setting it up, hope you like ops work.). You're doing auto-scaling (and that ASG? its not going to know about your pod resource requests, so you either need some very smart manual coordination between the two, or you need to set up cluster-autoscaler). You're setting up cluster metrics (definitely need metrics-server. not heapster, that was last year, metrics-server is this year. but how to visualize? do i host grafana in the cluster? then i need to worry about authn. cloudwatch really isn't made for these kinds of things... maybe I'll just give datadog a few thousand bucks.) Crap, 1.16 is out already? They only support 9 months of releases with security updates?! I feel like I just upgraded my nodes! Oh well, time to lose a day replicating this update across all of my environments. Haha - I had to chuckle on this. It really is this bad on EKS. My god, upgrades are a total joke. As bad as it is you have to believe that AWS seems to want to ignore Kubernetes and lock you in with ECS. They need to get EKS as easy to manage as GKE and they need to provide free control plane/masters like all the other managed k8s services. Hopefully we see something at Re:invent this year... at this point there is no reason to use EKS unless you are trapped into AWS. Which unfortunately is what AWS is counting on :(