3 ms·

Has anybody tried GKE and EKS from Amazon? What's your opinion on the later?

programd 8y ago
Here's a nice recent blog post about various issues you have to deal with to even set up a EKS cluster in AWS: https://medium.com/@dmaas/setting-up-amazon-eks-what-you-must-know-9b9c39627fbc https://medium.com/@dmaas/setting-up-amazon-eks-what-you-mus... I get the impression that compared to GKE, EKS is not yet fully baked as an easy to use product. It still involves too much manual fiddling around (e.g. see above) and operational complexity. I'd advise any clients of mine who want to run Kubernetes in the cloud to stick to GKE for now. The lower prices on GKE are a nice bonus also. That said, I don't discount what Amazon can do in this space. If they make the setup/admin experience as easy as spinning up a VM on AWS then Google will have a serious competitor. But not yet. BTW, for smaller shops and solo devs, I hear good things about the upcoming Digital Ocean Kubernetes service. It's in beta right now: https://www.digitalocean.com/products/kubernetes/ https://www.digitalocean.com/products/kubernetes/
- tofflos 8y ago
  I've spent the past two weeks trying to get Gitlab and Amazon EKS up and running and that blog post is pretty much spot on. I'm actually halfway through exploring some of the ingress options discussed in the post but I think I'm just going to try another provider now. Thanks for posting.
- runeks 8y ago
  > BTW, for smaller shops and solo devs, I hear good things about the upcoming Digital Ocean Kubernetes service. How would smaller shops and/or solo devs benefit from using Digital Ocean Kubernetes service over GKE?
ilikerashers 8y ago
I recently had a deep dive on EKS as part of an architectural review for a large scale (200+) microservice deployment. The recommendation was to strongly avoid EKS. Bad enough that the platform is brittle and expensive, the Amazon engineers sneered at Kubernetes as a "fad" and pushed us heavily towards ECS. I put it down to hubris but not a hope in hell we'd put our services onto a platform built with that attitude.
marcinzm 8y ago
I've used it for the last month+ now and it's not quiet there yet. Setup via terraform was mostly fine although the default subnet sizes were too small (since the networking layer seems to over-allocate IPs you only get 30% of the IPs usable for pods). The metrics backend is broken so metrics based auto-scaling of pods doesn't work. Also, r5 nodes aren't supported by the networking plugin since someone forgot to add them into it. User authentication for access to EKS is somewhat awkward. Also, the user role that creates the EKS cluster has magic admin access which I can't find a way to control.
manigandham 8y ago
EKS is basically half-managed. They originally had a better setup for master nodes but GKE has caught up with free masters and regional deployments (1 per zone) for the same high-availability. EKS masters will still run most of the cluster services to free up your worker nodes, but that's because they have very poor control over everything other than the masters. And since that's the most important part of the cluster, you will run into a lot of manual work and broken systems getting the cluster up and running on EKS. GKE is about 10x smoother and everything works as expected, and they are rapidly improving things with much better storage and networking options coming soon. At that point the experience will become 100x better than EKS.
runeks 8y ago
The recent thread about Kubernetes on Azure has some experience reports for EKS as well: https://news.ycombinator.com/item?id=17700360 https://news.ycombinator.com/item?id=17700360

Here's a nice recent blog post about various issues you have to deal with to even set up a EKS cluster in AWS: https://medium.com/@dmaas/setting-up-amazon-eks-what-you-must-know-9b9c39627fbc https://medium.com/@dmaas/setting-up-amazon-eks-what-you-mus... I get the impression that compared to GKE, EKS is not yet fully baked as an easy to use product. It still involves too much manual fiddling around (e.g. see above) and operational complexity. I'd advise any clients of mine who want to run Kubernetes in the cloud to stick to GKE for now. The lower prices on GKE are a nice bonus also. That said, I don't discount what Amazon can do in this space. If they make the setup/admin experience as easy as spinning up a VM on AWS then Google will have a serious competitor. But not yet. BTW, for smaller shops and solo devs, I hear good things about the upcoming Digital Ocean Kubernetes service. It's in beta right now: https://www.digitalocean.com/products/kubernetes/ https://www.digitalocean.com/products/kubernetes/

tofflos 8y ago
I've spent the past two weeks trying to get Gitlab and Amazon EKS up and running and that blog post is pretty much spot on. I'm actually halfway through exploring some of the ingress options discussed in the post but I think I'm just going to try another provider now. Thanks for posting.
runeks 8y ago
> BTW, for smaller shops and solo devs, I hear good things about the upcoming Digital Ocean Kubernetes service. How would smaller shops and/or solo devs benefit from using Digital Ocean Kubernetes service over GKE?

ilikerashers 8y ago

I recently had a deep dive on EKS as part of an architectural review for a large scale (200+) microservice deployment. The recommendation was to strongly avoid EKS. Bad enough that the platform is brittle and expensive, the Amazon engineers sneered at Kubernetes as a "fad" and pushed us heavily towards ECS. I put it down to hubris but not a hope in hell we'd put our services onto a platform built with that attitude.

marcinzm 8y ago

I've used it for the last month+ now and it's not quiet there yet. Setup via terraform was mostly fine although the default subnet sizes were too small (since the networking layer seems to over-allocate IPs you only get 30% of the IPs usable for pods). The metrics backend is broken so metrics based auto-scaling of pods doesn't work. Also, r5 nodes aren't supported by the networking plugin since someone forgot to add them into it. User authentication for access to EKS is somewhat awkward. Also, the user role that creates the EKS cluster has magic admin access which I can't find a way to control.

manigandham 8y ago

EKS is basically half-managed. They originally had a better setup for master nodes but GKE has caught up with free masters and regional deployments (1 per zone) for the same high-availability. EKS masters will still run most of the cluster services to free up your worker nodes, but that's because they have very poor control over everything other than the masters. And since that's the most important part of the cluster, you will run into a lot of manual work and broken systems getting the cluster up and running on EKS. GKE is about 10x smoother and everything works as expected, and they are rapidly improving things with much better storage and networking options coming soon. At that point the experience will become 100x better than EKS.

runeks 8y ago

The recent thread about Kubernetes on Azure has some experience reports for EKS as well: https://news.ycombinator.com/item?id=17700360 https://news.ycombinator.com/item?id=17700360