4 ms·
So you want to be a devops hero? Obtain google borg inspired container scheduling and management? It's a powerful abstraction, and I've enjoyed using K8s over
by nvarsj 10y ago
So you want to be a devops hero? Obtain google borg inspired container scheduling and management? It's a powerful abstraction, and I've enjoyed using K8s over the last year. However, like most freely available software, there is work to be done.
Here is my take on the state of things:
1. Still very much in rapid development, with features coming at a breakneck pace. I think Kubernetes is sold as production ready a bit too hard - it takes a good amount of effort to make a cluster production ready for anything non-trivial. Expect to contribute PRs to fix the issues you run in. It's probably easiest to use GCE since this has the most active development, I think. Otherwise, Openshift (Redhat's production fork) for on-premise installation. AWS is in a somewhat working state, with many improvements coming (what I'm using).
2. The maintainers are very open to contributions and discussion. PRs are generally accepted within a couple weeks (and given the volume of issues/PRs this is quite amazing). kubernetes.slack.com is a great way to talk to many of the core developers. I wish they had a slack subscription, so we could search the very rich chat history. A lot of stuff is missed unless you read the chat rooms daily.
3. My impression, which could be totally wrong I admit, is that much of the discussion for big changes seems to happen among the core Google & Redhat engineers offline. I wish the project used an open mailing list for these discussions, or did it in Slack (with a slack subscription, so history is available), or some other recorded text means. I don't think the current mechanisms scale to the size of the project and needs of the users.
3b. A lot of time seems devoted to things like scaling to 1000+ nodes, when fundamental things like kube-proxy are broken for basic use cases, in my opinion (kube-proxy uses an iptables hack for VIPs which leads to problems like the connection tracking table filling up and broken keepalive connections during deployments).
4. The provisioning scripts provided were traditionally pretty poor, kube-addons is a broken shell script mess. Fortunately, this is improving quickly. kops/kubeadm help with the provisioning, coreos is doing lots of work here as well, daemonsets to replace kube-addons, and so on. So this area is improving quickly. But expect to do work here if you're serious about using Kubernetes for prod. I'm using ansible/terraform for this.
5. Ingress (external access to your cluster), is still adhoc. This requires custom dev and testing to setup.
- swozey 10y agoJust a few comments. 1. GKE (Google Container Engine, GCE is compute) lags weeks behind any k8s release so if you're looking for bleeding edge it really isn't the best system, you're better off managing the cluster yourself. GKE being bumped to a recent version seems to be a private matter decided by a few Google employees, so you're never sure at all when you're going to get 1.37 or whatever version that is multiple sem-minors behind. It seems like it's usually about 2-3 weeks but I've never found any discussion or issues about its status so I wind up just checking every few days to see if it's been released. I have a mission critical feature that's being released (I hope) with 1.5 (IP persistence for sockets) and it'd be really nice to be able to follow the decision making process. Or at least have some understanding of what it entails (3 weeks of no major issues? GKE specific bugs? someone comfortable enough to release the hounds?) 5. I've had nothing but headaches with Ingress deployments. I'd wager it's by far the most brought up topic on the k8s slack. The documentation is all over the place and it seems like every example is completely different from the last, albeit there's never any explanation as to why exampleA is different from exampleB but does the same thing. Then you throw in annotations that might be required and IIRC unless you're reading the go docs you wouldn't know anything about them.
- thockingoog 10y agoGKE is currently 4 days behind OSS. That's hardly "weeks behind". 1.4.0 is available in GKE right now.
- swozey 10y agoYeah, that's a recent change and much appreciated. I was on 1.36 for quite awhile. The milestone mentioning the release dates to various regions is great, too. Not sure if that's always been there and I overlooked it, though.