4 ms·

FWIW kops gave my team a better kube experience on AWS than EKS (their managed version) guessing EKS isn't as bad now as it was 12 months ago, but 'managed kub

FWIW kops gave my team a better kube experience on AWS than EKS (their managed version) guessing EKS isn't as bad now as it was 12 months ago, but 'managed kube' varies tremendously across clouds, especially re ingress & overlay networks in my experience of AWS EKS, GCE's GKE and digitalocean's one, GCE has the tightest integration between load balancers and ingress, and even that can be clunky kops is overlay agnostic and lets you choose, which is a con but also a pro my guess is anyone that needs serious networking performance is still not sold on kube

skube 7y ago
my guess anyone needing serious anything is still not sold on kube seriously - why would you use k8s? it's basically a MLM scam ran by google
thinkingkong 7y ago
Out of curiosity what about a managed kube installation didnt work for your team?
- marcinzm 7y ago
  We use EKS. It's been buggy especially in the networking plugin (CNI). For example, version 1.5.4 released at the end of September caused ip rules to randomly disappear and break things in mysterious ways. Anyone who launched a cluster during the 4 days it took them to change the default had to manually downgrade their version of CNI. Of course, EKS doesn't provide a way to control the version of components they add to new clusters. Or, as I remember, the master account that is given initial full access to the cluster.
  - awinter-py 7y ago
    yup CNI
  - marcinzm 7y ago
    More broadly it just has so many rough edges and unfinished things. You can now do audit logging but you can't set an audit configuration so it logs everything (I think we had 200k events a day on a tiny cluster). You can tie service accounts to IAM roles but only if you cluster was created or upgraded after a certain date. You can't change or add subnets to the cluster without recreating it. Can't back a cluster or restore a cluster. Private load balancers require a non-standard configuration. EKS is still on 1.14 even though 1.17 is already out. The list goes on. Here's the current list of big and small issues people have with EKS: https://github.com/aws/containers-roadmap/issues?q=is%3Aopen+is%3Aissue+label%3AEKS https://github.com/aws/containers-roadmap/issues?q=is%3Aopen...
jjjensen90 7y ago
My enterprise and teams new to k8s have been very happy with Rancher on k3s with Postgres for racher's data plane, then just cloud providers configured (AWS in our case). It's very easy to deploy new clusters on AWS (we use terraform to deploy rancher and the rancher managed clusters) managed by rancher, have had zero issues with cloud provider resources integrations. Almost none of the engineers or devops team have kubernetes experience and we've been very happy with Rancher/RKE versus our previous experience with EKS.
nodesocket 7y ago
Heads up, AWS has a public GitHub repo (https://github.com/aws/containers-roadmap https://github.com/aws/containers-roadmap) where users can submit bugs and feature request for managed Kubernetes (EKS). It is quite robust and actively maintained by AWS engineers. Recently managed Kubernetes on AWS (EKS) rolled out the ability to make your master public or private. Previously if you set it to private, you were only able to lookup the master endpoint DNS entry within the VPC. After much backlash[1], they now advertise the private ip publically which allows much greater flexibility. A highly anticipated and nice change. [1] https://github.com/aws/containers-roadmap/issues/221 https://github.com/aws/containers-roadmap/issues/221
- awinter-py 7y ago
  making an open source contribution to benefit a big 5 company with deep networking expertise in-house and which also has forked secret versions of mysql and postgres should at least earn me some kind of rebate
  - nodesocket 7y ago
    huh? Rebate how you figure.
    - awinter-py 7y ago
      oh wait, this is just a bug tracker -- I thought you were suggesting people make OSS contributions to kube plugins that only work on EKS

skube 7y ago

my guess anyone needing serious anything is still not sold on kube seriously - why would you use k8s? it's basically a MLM scam ran by google

thinkingkong 7y ago

Out of curiosity what about a managed kube installation didnt work for your team?

marcinzm 7y ago
We use EKS. It's been buggy especially in the networking plugin (CNI). For example, version 1.5.4 released at the end of September caused ip rules to randomly disappear and break things in mysterious ways. Anyone who launched a cluster during the 4 days it took them to change the default had to manually downgrade their version of CNI. Of course, EKS doesn't provide a way to control the version of components they add to new clusters. Or, as I remember, the master account that is given initial full access to the cluster.
- awinter-py 7y ago
  yup CNI
- marcinzm 7y ago
  More broadly it just has so many rough edges and unfinished things. You can now do audit logging but you can't set an audit configuration so it logs everything (I think we had 200k events a day on a tiny cluster). You can tie service accounts to IAM roles but only if you cluster was created or upgraded after a certain date. You can't change or add subnets to the cluster without recreating it. Can't back a cluster or restore a cluster. Private load balancers require a non-standard configuration. EKS is still on 1.14 even though 1.17 is already out. The list goes on. Here's the current list of big and small issues people have with EKS: https://github.com/aws/containers-roadmap/issues?q=is%3Aopen+is%3Aissue+label%3AEKS https://github.com/aws/containers-roadmap/issues?q=is%3Aopen...

jjjensen90 7y ago

My enterprise and teams new to k8s have been very happy with Rancher on k3s with Postgres for racher's data plane, then just cloud providers configured (AWS in our case). It's very easy to deploy new clusters on AWS (we use terraform to deploy rancher and the rancher managed clusters) managed by rancher, have had zero issues with cloud provider resources integrations. Almost none of the engineers or devops team have kubernetes experience and we've been very happy with Rancher/RKE versus our previous experience with EKS.

nodesocket 7y ago

Heads up, AWS has a public GitHub repo (https://github.com/aws/containers-roadmap https://github.com/aws/containers-roadmap) where users can submit bugs and feature request for managed Kubernetes (EKS). It is quite robust and actively maintained by AWS engineers. Recently managed Kubernetes on AWS (EKS) rolled out the ability to make your master public or private. Previously if you set it to private, you were only able to lookup the master endpoint DNS entry within the VPC. After much backlash[1], they now advertise the private ip publically which allows much greater flexibility. A highly anticipated and nice change. [1] https://github.com/aws/containers-roadmap/issues/221 https://github.com/aws/containers-roadmap/issues/221

awinter-py 7y ago
making an open source contribution to benefit a big 5 company with deep networking expertise in-house and which also has forked secret versions of mysql and postgres should at least earn me some kind of rebate
- nodesocket 7y ago
  huh? Rebate how you figure.
  - awinter-py 7y ago
    oh wait, this is just a bug tracker -- I thought you were suggesting people make OSS contributions to kube plugins that only work on EKS