7 ms·

> Azure Kubernetes Service - Managed Kubernetes service, free cluster management Since this one is the bane of my existence at the moment. Yeah, this statemen

> Azure Kubernetes Service - Managed Kubernetes service, free cluster management Since this one is the bane of my existence at the moment. Yeah, this statement is technically correct, it is technically free, but you still pay for the VMs that are part of the cluster, and their disks. (Unless they can qualify for some other "free" metric.) But we've been using this, and we've had lots of spans of time where the managed API server will just time out requests. There's no guarantee on the free version; there's an SLO, but our metrics indicate that it isn't being met. (And our support requests were met with "upgrade to a paid SLA"… which we are, b/c we're a business, but if you're looking at this list, that presumably isn't you.)

arcticfox 6y ago
FWIW if anyone's looking, the hosted DigitalOcean Kubernetes clusters are fantastic. Despite being very early adopters, we've only had one issue and when we pointed the finger at them...the DO k8s team immediately diagnosed the issue, and it was our own fault. Woops.
- sofixa 6y ago
  Same with Scaleway's, both just work and are fully free. Some time ago i compared them against the competition and they fared very well even against GKE: https://atodorov.me/2020/06/14/comparing-kubernetes-managed-services-across-digital-ocean-scaleway-ovhcloud-and-linode/ https://atodorov.me/2020/06/14/comparing-kubernetes-managed-...
  - GordonS 6y ago
    Just want to says thanks for such an in-depth comparison!
  - deathanatos 6y ago
    I'm curious; do these host the control plane bits inside the cluster (on the VMs in the cluster, which I presume are not free)? In AKS, the API server & its backing storage, as well as most of the core controllers are "managed", in that they run somewhere not in your cluster. But then, you hit the issues where they must consume resources, and someone at the end of the day must pay for those. I'd almost rather they run in the cluster, since those are resources I can introspect & debug. (In a normal "vanilla" cluster, not a cloud-based one, normally these would all run inside the cluster, unless you configured them differently. E.g., kubeadm builds them into the cluster.)
    - sofixa 6y ago
      I think you're mistaken ( or i'm not understanding you correctly). In a default setup vanilla Kubernetes, you have master nodes which host the master services ( etcd, kube-apiserver, etc. etc.), and worker nodes on the side. With managed k8s, in most cases, the "control plane" ( what normally runs on the master nodes) is run by the cloud host, usually on shared instances ( Scaleway did a talk and it runs on dedicated Kubernetes clusters, in a namespace per client). Indeed, then you're subject to limitations ( e.g. the kube-apiserver for you is limited to 1000 MB of RAM and if you need more because you do X, you're screwed), but that's always the case - if you run your own master nodes, they have limitations as well ( based on instance/server size), just usually higher. There's the added risk that another customer might sature an unlimited resources and impact your control plane. In any case, I haven't hit any limitations with Scaleway's Kubernetes so far and haven't seen anyone talking about having hit one, while this isn't the first time i've heard about Digital Ocean. I don't know if it's due to size or because Scaleway have higher limits.
- maccard 6y ago
  Unfortunately DO only runs linux, and when a core piece of your infrastructure requires windows, it's a hard sell to say "well we'll just host _that bit_ on windows, and everything else on DO"...
- manishsharan 6y ago
  QQ: can DO k8s cluster scale up automatically with load ? And ,more importantly, can it scale down automatically when the load goes down.
  - ztjio 6y ago
    Neither of those functions are something Kubernetes does itself at all. For clusters to have that kind of functionality it's usually implemented with something like cluster-autoscaler. If it's not implemented with cluster autoscaler there's a chance you have a really annoying experience when nodes go away since the orchestrator doesn't know and takes a while to decide the node is gone and things should be cleaned up/rescheduled. Now that info aside, they do support cluster autoscaler. https://www.digitalocean.com/docs/kubernetes/how-to/autoscale/ https://www.digitalocean.com/docs/kubernetes/how-to/autoscal... Or maybe it's more appropriate to say that Cluster Autoscaler supports them? This tool (CA) does scaling in both directions and is highly configurable. At least, it is for other cloud vendors so I'd guess it's going to be so here too.
- stepbeek 6y ago
  Totally agree. I have a DO K8S cluster that we've supported since launch and I've had a single problem that was fixed by replacing a node. The Azure K8S cluster we support causes problems monthly. The client was heavily invested in an MS stack already so we picked Azure but we're likely going to get rid of it in in favour of paying a little bit more for SQL Server on AWS RDS in future.
davidhowlett 6y ago
My company has also hit capacity issues with the managed Azure Kubenetes that were not fixable as Azure users.
pbalau 6y ago
Aren't you missing the point a bit? I read that as "you don't have to pay someone to take care of your kubernetes cluster". In regular setups, you pay for the resources, then you pay someone to take care of those resources. In the Azure's case, and AWS EKS and maybe google offering too, you don't have to pay that engineer or team of engineers.
nikon 6y ago
Was an early adopter of AKS (after using the equally awful ACS) and wouldn't touch it with a barge pole. Remember being on engineering calls on a Friday evening with Microsoft engineers trying to explain the Kube master was completely broken. Volumes took 15 minutes to provision. Awful. P.s. your first GKE cluster is free

arcticfox 6y ago

FWIW if anyone's looking, the hosted DigitalOcean Kubernetes clusters are fantastic. Despite being very early adopters, we've only had one issue and when we pointed the finger at them...the DO k8s team immediately diagnosed the issue, and it was our own fault. Woops.

sofixa 6y ago
Same with Scaleway's, both just work and are fully free. Some time ago i compared them against the competition and they fared very well even against GKE: https://atodorov.me/2020/06/14/comparing-kubernetes-managed-services-across-digital-ocean-scaleway-ovhcloud-and-linode/ https://atodorov.me/2020/06/14/comparing-kubernetes-managed-...
- GordonS 6y ago
  Just want to says thanks for such an in-depth comparison!
- deathanatos 6y ago
  I'm curious; do these host the control plane bits inside the cluster (on the VMs in the cluster, which I presume are not free)? In AKS, the API server & its backing storage, as well as most of the core controllers are "managed", in that they run somewhere not in your cluster. But then, you hit the issues where they must consume resources, and someone at the end of the day must pay for those. I'd almost rather they run in the cluster, since those are resources I can introspect & debug. (In a normal "vanilla" cluster, not a cloud-based one, normally these would all run inside the cluster, unless you configured them differently. E.g., kubeadm builds them into the cluster.)
  - sofixa 6y ago
    I think you're mistaken ( or i'm not understanding you correctly). In a default setup vanilla Kubernetes, you have master nodes which host the master services ( etcd, kube-apiserver, etc. etc.), and worker nodes on the side. With managed k8s, in most cases, the "control plane" ( what normally runs on the master nodes) is run by the cloud host, usually on shared instances ( Scaleway did a talk and it runs on dedicated Kubernetes clusters, in a namespace per client). Indeed, then you're subject to limitations ( e.g. the kube-apiserver for you is limited to 1000 MB of RAM and if you need more because you do X, you're screwed), but that's always the case - if you run your own master nodes, they have limitations as well ( based on instance/server size), just usually higher. There's the added risk that another customer might sature an unlimited resources and impact your control plane. In any case, I haven't hit any limitations with Scaleway's Kubernetes so far and haven't seen anyone talking about having hit one, while this isn't the first time i've heard about Digital Ocean. I don't know if it's due to size or because Scaleway have higher limits.
maccard 6y ago
Unfortunately DO only runs linux, and when a core piece of your infrastructure requires windows, it's a hard sell to say "well we'll just host _that bit_ on windows, and everything else on DO"...
manishsharan 6y ago
QQ: can DO k8s cluster scale up automatically with load ? And ,more importantly, can it scale down automatically when the load goes down.
- ztjio 6y ago
  Neither of those functions are something Kubernetes does itself at all. For clusters to have that kind of functionality it's usually implemented with something like cluster-autoscaler. If it's not implemented with cluster autoscaler there's a chance you have a really annoying experience when nodes go away since the orchestrator doesn't know and takes a while to decide the node is gone and things should be cleaned up/rescheduled. Now that info aside, they do support cluster autoscaler. https://www.digitalocean.com/docs/kubernetes/how-to/autoscale/ https://www.digitalocean.com/docs/kubernetes/how-to/autoscal... Or maybe it's more appropriate to say that Cluster Autoscaler supports them? This tool (CA) does scaling in both directions and is highly configurable. At least, it is for other cloud vendors so I'd guess it's going to be so here too.
stepbeek 6y ago
Totally agree. I have a DO K8S cluster that we've supported since launch and I've had a single problem that was fixed by replacing a node. The Azure K8S cluster we support causes problems monthly. The client was heavily invested in an MS stack already so we picked Azure but we're likely going to get rid of it in in favour of paying a little bit more for SQL Server on AWS RDS in future.

davidhowlett 6y ago

My company has also hit capacity issues with the managed Azure Kubenetes that were not fixable as Azure users.

pbalau 6y ago

Aren't you missing the point a bit? I read that as "you don't have to pay someone to take care of your kubernetes cluster". In regular setups, you pay for the resources, then you pay someone to take care of those resources. In the Azure's case, and AWS EKS and maybe google offering too, you don't have to pay that engineer or team of engineers.

nikon 6y ago

Was an early adopter of AKS (after using the equally awful ACS) and wouldn't touch it with a barge pole. Remember being on engineering calls on a Friday evening with Microsoft engineers trying to explain the Kube master was completely broken. Volumes took 15 minutes to provision. Awful. P.s. your first GKE cluster is free