4 ms·

AWS in my mind can quickly lose the kubernetes war amongst cloud providers. This is every cloud providers chance: EKS on AWS is so damn tied into a bunch of oth

AWS in my mind can quickly lose the kubernetes war amongst cloud providers. This is every cloud providers chance: EKS on AWS is so damn tied into a bunch of other AWS products that it's literally impossible to just delete a cluster now. I tried. It's tied into VPCs and Subnets and EC2 and Load balancers and a bunch of other products that no longer makes sense now that K8s won. In my opinion it needs to be re-engineered completely into a super slim product that is not tied to all these crazy things.

ffo 5y ago
You mean EKS needs re-engineering?
kinghajj 5y ago
Also the fact that a new EKS cluster takes at least 20 minutes to come up and be ready, makes AWS' offering the weakest among the Big Three cloud vendors.
- jen20 5y ago
  This isn’t true. I’ve provisioned 53 EKS clusters this week, and every one of them has been up in under 11 minutes with all of the accoutrements. I understand it has been substantially slower in the past.
  - nickjj 5y ago
    11 minutes is still a long time IMO. You can spin up a local multi-node cluster using kind[0] in 1 minute on 6+ year old hardware. I know it's not the same but I really have to imagine there's ways to speed this up on the cloud. I haven't spun up a cluster on DO or Linode in a while, does anyone know how long it takes there for a comparison? [0]: https://kind.sigs.k8s.io/ https://kind.sigs.k8s.io/
    - mlnj 5y ago
      I think it is unfair to compare KinD with an actual Kubernetes cluster which comes with Load Balancers, External IP addresses etc. My Terraform scripts get a HA K3s cluster in Google Cloud VMs in less than 9 minutes, which in my opinion is fantastic.
      - BenTheElder 5y ago
        Those things don't have to be inherently slow and e.g. external IP don't need to block bring-up. I've worked on KIND and on clusters on clouds (at Google, but on multiple clouds) and both can be very quick, if anything there's still low hanging fruit to make KIND faster that I'd expect a production service with more staffing to handle. KIND is Kubernetes, typically on much weaker hardware :-) Within a few minutes is a perfectly reasonable expectation even for "real"-er clusters, see e.g. under 4 minutes: - https://kubedex.com/google-gke-vs-azure-aks-automation-and-reliability/ https://kubedex.com/google-gke-vs-azure-aks-automation-and-r... - https://stackoverflow.com/questions/53839292/is-a-3-minute-gke-cluster-startup-time-normal https://stackoverflow.com/questions/53839292/is-a-3-minute-g...
  - misiti3780 5y ago
    Are you using terraform ?
    - jen20 5y ago
      A variety of Pulumi, Terraform and eksctl.
  - nickjj 5y ago
    I just spun up a very simple 3 node EKS cluster using eksctl on us-east-1 and it took 19 minutes. Still seems quite slow to me, I'm not sure how you got 11 minutes over a sample size of 50+ clusters, maybe it's dependent on region or what worker node types you used?
- hughrr 5y ago
  Also the base cluster cost of $0.10 an hour really hurts for test and experimentation.
  - jrockway 5y ago
    GCP charges the same now (though your first one is free). I don't think it's an unreasonable cost, but it's certainly not competitive with places like Digital Ocean which are still free.
- hodgesrm 5y ago
  It really depends on the use case. We use Kubernetes for hosting managed data warehouses. EKS cluster spin-up time is a one-time cost to start a new environment for data warehouses. It's insignificant compared to the time required to load data. Other issues like auto-scaling performance are more important, at least in our case.
unfunco 5y ago
It's not literally impossible to delete a cluster, I do this many times daily fully automated with no issues, and an EKS cluster is not tied to a VPC or subnets, you can spin them up independently, and you can delete your clusters without affecting the VPC or subnets in any way. An Ingress or a service of type LoadBalancer will create a load balancer in AWS that's tied to your cluster, but that's the whole point of Kubernetes, it'll spin up the equivalent resources in Azure or GCP or DigitalOcean.
k__ 5y ago
I don't have the impression AWS even sees it as a war worth fighting. And I can't blame them.
misiti3780 5y ago
I am about to move our infrastructure from ECS to EKS, what other AWS products is EKS tied to?
atmosx 5y ago
Not sure. I'm using DigitalOcean Managed Version and have experience with EKS. There are pros and cons on both sides. What I like about EKS is support for API and k8s logs (accounting / security), coupled with CloudTrail and IAM integration for RBAC/user management is a bliss.

ffo 5y ago

You mean EKS needs re-engineering?

kinghajj 5y ago

Also the fact that a new EKS cluster takes at least 20 minutes to come up and be ready, makes AWS' offering the weakest among the Big Three cloud vendors.

jen20 5y ago
This isn’t true. I’ve provisioned 53 EKS clusters this week, and every one of them has been up in under 11 minutes with all of the accoutrements. I understand it has been substantially slower in the past.
- nickjj 5y ago
  11 minutes is still a long time IMO. You can spin up a local multi-node cluster using kind[0] in 1 minute on 6+ year old hardware. I know it's not the same but I really have to imagine there's ways to speed this up on the cloud. I haven't spun up a cluster on DO or Linode in a while, does anyone know how long it takes there for a comparison? [0]: https://kind.sigs.k8s.io/ https://kind.sigs.k8s.io/
  - mlnj 5y ago
    I think it is unfair to compare KinD with an actual Kubernetes cluster which comes with Load Balancers, External IP addresses etc. My Terraform scripts get a HA K3s cluster in Google Cloud VMs in less than 9 minutes, which in my opinion is fantastic.
    - BenTheElder 5y ago
      Those things don't have to be inherently slow and e.g. external IP don't need to block bring-up. I've worked on KIND and on clusters on clouds (at Google, but on multiple clouds) and both can be very quick, if anything there's still low hanging fruit to make KIND faster that I'd expect a production service with more staffing to handle. KIND is Kubernetes, typically on much weaker hardware :-) Within a few minutes is a perfectly reasonable expectation even for "real"-er clusters, see e.g. under 4 minutes: - https://kubedex.com/google-gke-vs-azure-aks-automation-and-reliability/ https://kubedex.com/google-gke-vs-azure-aks-automation-and-r... - https://stackoverflow.com/questions/53839292/is-a-3-minute-gke-cluster-startup-time-normal https://stackoverflow.com/questions/53839292/is-a-3-minute-g...
- misiti3780 5y ago
  Are you using terraform ?
  - jen20 5y ago
    A variety of Pulumi, Terraform and eksctl.
- nickjj 5y ago
  I just spun up a very simple 3 node EKS cluster using eksctl on us-east-1 and it took 19 minutes. Still seems quite slow to me, I'm not sure how you got 11 minutes over a sample size of 50+ clusters, maybe it's dependent on region or what worker node types you used?
hughrr 5y ago
Also the base cluster cost of $0.10 an hour really hurts for test and experimentation.
- jrockway 5y ago
  GCP charges the same now (though your first one is free). I don't think it's an unreasonable cost, but it's certainly not competitive with places like Digital Ocean which are still free.
hodgesrm 5y ago
It really depends on the use case. We use Kubernetes for hosting managed data warehouses. EKS cluster spin-up time is a one-time cost to start a new environment for data warehouses. It's insignificant compared to the time required to load data. Other issues like auto-scaling performance are more important, at least in our case.

unfunco 5y ago

It's not literally impossible to delete a cluster, I do this many times daily fully automated with no issues, and an EKS cluster is not tied to a VPC or subnets, you can spin them up independently, and you can delete your clusters without affecting the VPC or subnets in any way. An Ingress or a service of type LoadBalancer will create a load balancer in AWS that's tied to your cluster, but that's the whole point of Kubernetes, it'll spin up the equivalent resources in Azure or GCP or DigitalOcean.

k__ 5y ago

I don't have the impression AWS even sees it as a war worth fighting. And I can't blame them.

misiti3780 5y ago

I am about to move our infrastructure from ECS to EKS, what other AWS products is EKS tied to?

atmosx 5y ago

Not sure. I'm using DigitalOcean Managed Version and have experience with EKS. There are pros and cons on both sides. What I like about EKS is support for API and k8s logs (accounting / security), coupled with CloudTrail and IAM integration for RBAC/user management is a bliss.