4 ms·

1. Reliable rolling deployments. We didn't have these before. Yes, you can implement them without K8s (I have, at other companies) but to get the full set of f

1. Reliable rolling deployments. We didn't have these before. Yes, you can implement them without K8s (I have, at other companies) but to get the full set of features K8s provides, such as deploying N services in parallel, taking no more than X% of your capacity offline at a time, short-circuiting in the event the app is dead on arrival, connection draining with timeouts, you end up with a VERY complicated multi-threaded codebase. 2. Seamless horizontal scale-out. Want to scale up your app in a test environment from 3 replicas to 6 to do some performance testing? This used to be a DevOps ticket that would take a few days -- DevOps engineer tries running Terraform, but oh no, a CentOS package update seems to have broken our Puppet manifests and we have to fix that. Now the developer makes a PR to the GitOps repo where they adjust a single YAML setting. 3. GitOps/ArgoCD. ArgoCD is possibly my favorite piece of software of all time. It provides incredible visualizations for what's happening with your Kubernetes infrastructure. It really increases your confidence and trust in the system to be able watch a deployment rollout or scaling operation happen in real time. ArgoCD makes it spectacularly obvious when something has gone wrong -- you still sometimes need to go spelunking through the GCP console or use kubectl to inspect resources, but to a much lesser degree. I cannot emphasize enough how magical it is. These are all things that our initial implementation has delivered, we are also planning to try to leverage K8s for things like on-demand pull request preview environments, hosted developer environments, and canary deploys that are MUCH harder to implement in a world without K8s (trust me, I've done it).

Spivak 6y ago
You’re using hosted k8s! That explains everything! I was so confused how a devops person wasn’t cursing the name. Implementing and managing k8s is where all the complexity and headache lives.
- anchochilis 6y ago
  Updated original comment to clarify :) Although, to be fair, the article is about GKE.
- anthonyskipper 6y ago
  If your company is implementing and managing their own k8s clusters they are doing it wrong. Use a cloud hosted solution.
- sontek 6y ago
  Have you seen kops? rancher? loft? I don't think its that terrible to manage k8s at all using the tools available to you. k8s definitely used to be be difficult to manage but that isn't the case anymore. But literally every cloud provider out there has a managed solution so at this point you really only need to do it for DC work or if you like to do it. Amazon - EKS Google - GKE Azure - AKS anything beyond those 3 is a rounding error but... Linode - LKE DigitalOcean - DigitalOcean Kubernetes IBM Cloud Kubernetes Service Rackspace - KAAS
  - Spivak 6y ago
    I've used Rancher and while it is very nice it still punts a lot of the hard bits like interacting with the outside network, vm creation, and storage (especially shared persistent storage) to the ops team to figure out. Creating a big-ole cluster of app servers was never really the hard ops problem which is what k8s does really well.
    - sontek 6y ago
      Creating the cluster of servers isn't the value of kubernetes. We all had clusters of servers well before k8s, openstack, and things like AWS. The benefit of k8s is the orchestration of those clusters. Spinning up 6 new http servers and getting them added to the load balancer automatically. Generating 4 new memcached nodes and getting them registered in DNS so clients pick them up and add them to the hashring. The benefit of k8s is the scaling and elastic capabilities. It can trigger vertical scaling by spinning up larger pods or horizontal scaling by adding/removing pods. Anyone thinking that people are using k8s because it can create app servers doesn't understand why anyone is using k8s. If all we needed to do is create a cluster of app servers, we wouldn't be using k8s. That being said, a cluster of app servers still needs orchestration and config management and we had a ton of crazy solutions for that prior to k8s.
- mbushey 6y ago
  I previously managed my companies Kubernetes clusters using Kops on AWS. My company switched to Google, so I had to move everything to GKE. I miss Kops, it gave me more control and made life easier. I don't want to use the old crappy deprecated DNS server, but with GKE that's what you get. I don't get to control what goes on the masters on my clusters because they're not mine anymore. Having GKE do the masters means more headaches and complexity because I no longer have control. I love K8s, but GKE does a pretty poor job implementing it. It boggles my mind that the main company behind K8s is so incompetent in implementing it.
  - farisjarrah 6y ago
    Its a bit of a pain, but you can definitely switch to CoreDNS on GKE if you need to. Also as someone who has used GKE, EKS, MicroK8s, and Minikube, I would say that the easiest to use implementation of Kubenetes as a Service is GKE. Not saying its easy or the worlds most thought out product, but I gotta give google just a tiny bit of credit for at least being easier to use then the competition.
vp8989 6y ago
Yeh this list is not surprising and none of these things you mentioned exclusively require Kubernetes or running containers in Prod. Like I mentioned in another comment, it feels like the purpose of Kubernetes is that it's "brand" provides political cover to introduce these practices to the engineering org.
- anchochilis 6y ago
  I don't know what to tell you except that with years of experience implementing this infrastructure outside Kubernetes, letting Kubernetes handle it is cheaper. That doesn't mean Kubernetes is a good fit for every organization or workload. You can roll your own anything with enough time and manpower. Whether it makes sense to do so depends on your circumstances.
  - zug_zug 6y ago
    Well for example, I'm a lead devops on a team of 2-3. We have 6 core services. For the last several years, due to static load we've been fine with 2 instances per service. Now that we want autoscale (even though all the real load is really just the DB), it seems we could get ourselves onto AWS autoscale in ~1 month, though it would require some coding. Spending 3 devops on 1 year doing something is a red-flag to me. If your changes save devs 1 hour per deploy, it'll pay off 6,000 deploys from now.
    - anchochilis 6y ago
      It's not the only thing we've been doing, just the most important. I think it's great you guys have 2.5 dedicated people operating 6 services across 12 prod instances, with that kind of ratio our team would be 15 people!
    - justin_oaks 6y ago
      I agree with your point overall, I just want to add that some DevOps efforts aren't to save DevOps time, but to prevent errors. When you make a change manually, there's a chance that you forget something, have a typo, etc. Those problems disappear when those same things are automated.

Spivak 6y ago

You’re using hosted k8s! That explains everything! I was so confused how a devops person wasn’t cursing the name. Implementing and managing k8s is where all the complexity and headache lives.

anchochilis 6y ago
Updated original comment to clarify :) Although, to be fair, the article is about GKE.
anthonyskipper 6y ago
If your company is implementing and managing their own k8s clusters they are doing it wrong. Use a cloud hosted solution.
sontek 6y ago
Have you seen kops? rancher? loft? I don't think its that terrible to manage k8s at all using the tools available to you. k8s definitely used to be be difficult to manage but that isn't the case anymore. But literally every cloud provider out there has a managed solution so at this point you really only need to do it for DC work or if you like to do it. Amazon - EKS Google - GKE Azure - AKS anything beyond those 3 is a rounding error but... Linode - LKE DigitalOcean - DigitalOcean Kubernetes IBM Cloud Kubernetes Service Rackspace - KAAS
- Spivak 6y ago
  I've used Rancher and while it is very nice it still punts a lot of the hard bits like interacting with the outside network, vm creation, and storage (especially shared persistent storage) to the ops team to figure out. Creating a big-ole cluster of app servers was never really the hard ops problem which is what k8s does really well.
  - sontek 6y ago
    Creating the cluster of servers isn't the value of kubernetes. We all had clusters of servers well before k8s, openstack, and things like AWS. The benefit of k8s is the orchestration of those clusters. Spinning up 6 new http servers and getting them added to the load balancer automatically. Generating 4 new memcached nodes and getting them registered in DNS so clients pick them up and add them to the hashring. The benefit of k8s is the scaling and elastic capabilities. It can trigger vertical scaling by spinning up larger pods or horizontal scaling by adding/removing pods. Anyone thinking that people are using k8s because it can create app servers doesn't understand why anyone is using k8s. If all we needed to do is create a cluster of app servers, we wouldn't be using k8s. That being said, a cluster of app servers still needs orchestration and config management and we had a ton of crazy solutions for that prior to k8s.
mbushey 6y ago
I previously managed my companies Kubernetes clusters using Kops on AWS. My company switched to Google, so I had to move everything to GKE. I miss Kops, it gave me more control and made life easier. I don't want to use the old crappy deprecated DNS server, but with GKE that's what you get. I don't get to control what goes on the masters on my clusters because they're not mine anymore. Having GKE do the masters means more headaches and complexity because I no longer have control. I love K8s, but GKE does a pretty poor job implementing it. It boggles my mind that the main company behind K8s is so incompetent in implementing it.
- farisjarrah 6y ago
  Its a bit of a pain, but you can definitely switch to CoreDNS on GKE if you need to. Also as someone who has used GKE, EKS, MicroK8s, and Minikube, I would say that the easiest to use implementation of Kubenetes as a Service is GKE. Not saying its easy or the worlds most thought out product, but I gotta give google just a tiny bit of credit for at least being easier to use then the competition.

vp8989 6y ago

Yeh this list is not surprising and none of these things you mentioned exclusively require Kubernetes or running containers in Prod. Like I mentioned in another comment, it feels like the purpose of Kubernetes is that it's "brand" provides political cover to introduce these practices to the engineering org.

anchochilis 6y ago
I don't know what to tell you except that with years of experience implementing this infrastructure outside Kubernetes, letting Kubernetes handle it is cheaper. That doesn't mean Kubernetes is a good fit for every organization or workload. You can roll your own anything with enough time and manpower. Whether it makes sense to do so depends on your circumstances.
- zug_zug 6y ago
  Well for example, I'm a lead devops on a team of 2-3. We have 6 core services. For the last several years, due to static load we've been fine with 2 instances per service. Now that we want autoscale (even though all the real load is really just the DB), it seems we could get ourselves onto AWS autoscale in ~1 month, though it would require some coding. Spending 3 devops on 1 year doing something is a red-flag to me. If your changes save devs 1 hour per deploy, it'll pay off 6,000 deploys from now.
  - anchochilis 6y ago
    It's not the only thing we've been doing, just the most important. I think it's great you guys have 2.5 dedicated people operating 6 services across 12 prod instances, with that kind of ratio our team would be 15 people!
  - justin_oaks 6y ago
    I agree with your point overall, I just want to add that some DevOps efforts aren't to save DevOps time, but to prevent errors. When you make a change manually, there's a chance that you forget something, have a typo, etc. Those problems disappear when those same things are automated.