4 ms·

Did you experience the downtime issues with DigitalOcean managed Kubernetes that the author also experienced? I'm also considering DOKS so would be great to kno

Did you experience the downtime issues with DigitalOcean managed Kubernetes that the author also experienced? I'm also considering DOKS so would be great to know!

CraftThatBlock 6y ago
I had no issues so far, everything has been fairly smooth. Setup was very easy too (was my first cluster setup), and pricing is very good.
nrmitchi 6y ago
Note: source for this information is from discussions with DO via support and the kubernetes slack. I am not affiliated with DO in any way. The downtime issues with DOKS are directly related to the resources allocated to the control plane, which is not set up in a HA capacity. The resources assigned are directly related to the size/number of nodes you use. API-heavy applications can very easy knock out the control plane, and in that situation is takes it a (relatively) long time to recover on it's own (I would typically see ~4hr before it became responsive again). Their support team is able to modify the master resources for a given cluster (to assist with recovery), but the turn-around time on that shouldn't be considered "production ready". At this point my advice for DOKS would be: - Are you using very basic, out-of-the-box Kubernetes to host "apps"? You will probably be fine, but be sure to have a back-up. - Are you planning to use Operators, or anything that heavily interacts with the kube-api? I would recommend not using it, or over-provisioning your cluster (which would very quickly offset the advantage of the "free" managed masters). I know that they are working on fixing some of these reliability issues and I have hopes that it will be more stable shortly. At this time I have a "stable" cluster which (unless their support lied to me) had master resources manually increased by their support team after the 3rd incident of it dying. I haven't had an issue since then.
- tebbers 6y ago
  Thanks for the info. That doesn't sound good - one major reason for considering moving to Kubernetes was for the high availability.
  - nrmitchi 6y ago
    Availability of your applications, and availability of the kubernetes control plane, are not typically correlated. One of the nice things about how Kubernetes orchestrates is that the master is not in the path of requests. Ie, in many situations the Kubernetes master may be unavailable, but your applications will be unaffected and can still serve traffic.
    - tebbers 6y ago
      Thanks for the explanation!
- nickthemagicman 6y ago
  What do you mean API heavy? Do you mean hitting the kubernetes API? Does k8s auto scaling count?
  - amzans 6y ago
    I guess he means the Kubernetes API which is what most of the controllers, and other components talk to. That was my case too. It was mostly when I installed the prometheus-operator that things would go south, even if the nodes themselves were perfectly healthy and underutilized.
    - nickthemagicman 6y ago
      Gotcha, thanks for the info! Do you know if were just doing run of the mill Pod deploys of like wordpress and little MVC apps and stuff with no K8s api calls we should be fine? I like the autoscaling and the infrastructure as code, and price for DOKS is good.
      - amzans 6y ago
        I think that shouldn’t cause much troubles in DO. I had it running for a while without the prometheus operator, and I didn’t have any issue. In the beginning I was even running everything on a one $10/mo “cluster” (cheapest DO offers). Even Postgres and Redis, all inside the cluster. As I got more customers, I needed something less risky, so added more nodes, and moved the database out. But to be honest even the 1 instance $10 cluster was fine.
        nickthemagicman 6y ago
        Belated thanks for the info!

Did you experience the downtime issues with DigitalOcean managed Kubernetes that the author also experienced? I'm also considering DOKS so would be great to know!

CraftThatBlock 6y ago

I had no issues so far, everything has been fairly smooth. Setup was very easy too (was my first cluster setup), and pricing is very good.

nrmitchi 6y ago

Note: source for this information is from discussions with DO via support and the kubernetes slack. I am not affiliated with DO in any way. The downtime issues with DOKS are directly related to the resources allocated to the control plane, which is not set up in a HA capacity. The resources assigned are directly related to the size/number of nodes you use. API-heavy applications can very easy knock out the control plane, and in that situation is takes it a (relatively) long time to recover on it's own (I would typically see ~4hr before it became responsive again). Their support team is able to modify the master resources for a given cluster (to assist with recovery), but the turn-around time on that shouldn't be considered "production ready". At this point my advice for DOKS would be: - Are you using very basic, out-of-the-box Kubernetes to host "apps"? You will probably be fine, but be sure to have a back-up. - Are you planning to use Operators, or anything that heavily interacts with the kube-api? I would recommend not using it, or over-provisioning your cluster (which would very quickly offset the advantage of the "free" managed masters). I know that they are working on fixing some of these reliability issues and I have hopes that it will be more stable shortly. At this time I have a "stable" cluster which (unless their support lied to me) had master resources manually increased by their support team after the 3rd incident of it dying. I haven't had an issue since then.

tebbers 6y ago
Thanks for the info. That doesn't sound good - one major reason for considering moving to Kubernetes was for the high availability.
- nrmitchi 6y ago
  Availability of your applications, and availability of the kubernetes control plane, are not typically correlated. One of the nice things about how Kubernetes orchestrates is that the master is not in the path of requests. Ie, in many situations the Kubernetes master may be unavailable, but your applications will be unaffected and can still serve traffic.
  - tebbers 6y ago
    Thanks for the explanation!
nickthemagicman 6y ago
What do you mean API heavy? Do you mean hitting the kubernetes API? Does k8s auto scaling count?
- amzans 6y ago
  I guess he means the Kubernetes API which is what most of the controllers, and other components talk to. That was my case too. It was mostly when I installed the prometheus-operator that things would go south, even if the nodes themselves were perfectly healthy and underutilized.
  - nickthemagicman 6y ago
    Gotcha, thanks for the info! Do you know if were just doing run of the mill Pod deploys of like wordpress and little MVC apps and stuff with no K8s api calls we should be fine? I like the autoscaling and the infrastructure as code, and price for DOKS is good.
    - amzans 6y ago
      I think that shouldn’t cause much troubles in DO. I had it running for a while without the prometheus operator, and I didn’t have any issue. In the beginning I was even running everything on a one $10/mo “cluster” (cheapest DO offers). Even Postgres and Redis, all inside the cluster. As I got more customers, I needed something less risky, so added more nodes, and moved the database out. But to be honest even the 1 instance $10 cluster was fine.
      - nickthemagicman 6y ago
        Belated thanks for the info!