6 ms·

Kubernetes means you don't have to learn each clouds way of doing a deployment. You just learn the k8s way then use that with Google, Azure or whatever. So you

Kubernetes means you don't have to learn each clouds way of doing a deployment. You just learn the k8s way then use that with Google, Azure or whatever. So your skillset is reusable.

finikytou 3y ago
no it doesnt mean that. you still need to know how to operate k8s on a specific cloud.
- quickthrower2 3y ago
  hell yes.
- jml78 3y ago
  Just like you have to know how to operate each cloud in general. There is no free lunch. But if you learn k8s, moving from AWS EKS to Google GKE to DigitalOceans hosted k8s is easy.
- Art9681 3y ago
  No you don't have to. You can deploy your own cluster instead of using the managed option if you want to. A good SRE can deploy and manage EKS. A great SRE can deploy and manage a cluster to any Cloud without ever touching the dashboards.
- hhh 3y ago
  Ideally the developer doesn’t. At scale some platform or infra team should.
- artdigital 3y ago
  I run stuff on hosted k8s from DO, Google and Vultr. I can absolutely reuse my knowledge, and deployments are almost identical (minus smaller differences like storage csi driver, etc)
  - oceanplexian 3y ago
    I work at a place running a million containers deployed in all 3 (Azure, AWS, GCP). I can assure you they are radically different; autoscaling works differently, the load balancers work differently, the networking infrastructure is completely different, the failure modes and limits behaviors are different, the instances perform differently, observability is different, and they all suck in unique and different ways that we discover on a daily basis. Shit even AWS can't keep their regions consistent; each region has different products and features and they fail in different ways. If you are the one maintaining it it's a full time job handling all these edge cases, it's completely miserable and I wouldn't recommend it to anyone.
    - Infernal 3y ago
      > I work at a place running a million containers deployed in all 3 (Azure, AWS, GCP) Are you using AKS, EKS, GKE on those providers, or deploying your own k8s on top of the compute those providers offer? It sounds to me like the former.
      - hosh 3y ago
        I’ve done smaller deployments on GKE and another on EKS, and I can tell you, they are different enough. It’s when you start having to autoscale, optimize resources by instance types, and manage network ingress that these quirks start really come out. The essential ideas are invariant across cloud providers though. But I enjoy working with Kubernetes.
        Infernal 3y ago
        I should’ve been clearer about what I was getting at. I agree AKS, EKS, GKE etc (cloud gnostic k8s) are different enough to cause a growth of complexity when managing a mixed environment of them. The post I was replying to seemed to be saying (by analogy) “Linux is hard to manage because I run into all sorts of trouble trying to support a mixed environment of SuSE, Ubuntu and RHEL, therefore Linux is just too complicated”.
      - 3y ago
- okamiueru 3y ago
  Some part of it creates cloud specific resources, and you might also for good reasons have cloud manged database or data storage that your k8s services use. However, "k8s on a specific cloud", is mostly the same, except for the outer edges.
  - hosh 3y ago
    Until you want to scale and optimize resources. I enjoy working with Kubernetes, but forcing a complex domain into something legible is a recipe for catastrophe. There are quirks, across cloud providers, and this is just another day in Ops, with or without Kubernetes. (See: https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-called-legibility/ https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-call... )
- api 3y ago
  It’s more like learning different Linux distributions than learning different OSes.
jcims 3y ago
Kubernetes gets a lot of shit on HN but for all of its challenges it has proven to be a fantastic method to abstract many of the idiosyncrasies of hosting on-prem vs various cloud providers. I've worked for two companies now with 8 figure monthly cloud spend and hundreds/thousands of applications operating in one or more of the main cloud providers and k8s has been essential in making that happen. Teams can migrate to an on-premise hosted option if they want, then transition to cloud if/when it makes sense, or just stay where they are.
- stavros 3y ago
  Takes notes "Kubernetes has been essential in making 8-figure monthly cloud spend happen"
  - windexh8er 3y ago
    The last startup I was in was obsessed with putting everything into k8s for no apparent reason. Even the product they were selling, which most customers hated because it forced our customer base to either have to deal with the pain of paying for k8s because they had no need or intention to use it other than for our product or work with a cross-functional team which now created a time sink and dependency that wouldn't have been there otherwise. The best though was when I ran across someone in the org trying to run a single container to run a periodic job in its own cluster. They spent half the day trying to get it to work with ingress. You can imagine how it came to a head when the company realized they were spending hundreds of thousands per month on idle clusters in AWS.
    - immibis 3y ago
      One company I worked at was obsessed with k8s for a while, on a local arrangement of about 4 servers, each build would start up a new container on kubernetes and rebuild an entire operating system from scratch.
      - immibis 3y ago
        clarification: the thing being built was an operating system. It didn't build an operating system to run the real build on.
oceanplexian 3y ago
> Kubernetes means you don't have to learn each clouds way of doing a deployment. So instead of learning how to deploy on GCP, AWS, and Azure, which is only 3x more complicated than deploying to a single cloud, you should learn K8s, which is 10-15x more complicated, in addition to still having to learn about all the various ingress controllers and weird quirks that are completely different on each cloud provider. Doesn't really track for me.
- echelon 3y ago
  > which is 10-15x more complicated You can learn k8s in a day. It's really simple. > various ingress controllers and weird quirks that are completely different on each cloud provider Which are thoroughly documented and not that hard to implement or understand. You'd be reading about each cloud's nonstandard ingress even without k8s. The beauty of k8s is you can run your software locally and have a much easier time lifting and shifting to another cloud. Fitting to the shape of a cloud provider is a great way to never leave. Another benefit of k8s is that you treat your services as cattle you can easily spawn and kill. Adoption of k8s naturally leads to anti-fragility, anti-brittle best practices.
  - hosh 3y ago
    I think being able to abstract the cloud providers is secondary to k8s’s ability to self-heal and its modularity. But I think most developers don’t care, and instead, should interact with a platform built with Kubernetes as a foundation.
  - profunctor 3y ago
    Maybe this is because I’m not that smart but I could not learn real kubernetes in a day. I had to build a system for loading models and returning predictions over a HTTPS api. It had to connect to storage to load the model, needed secrets etc. It took more than a day. And I think it would take most people more than a day to go from zero to able to create a useful, real world deployment in a day. I’m sure you can rush through the documentation in a day but I wouldn’t call that learning.
  - bg24 3y ago
berkle4455 3y ago
This argument is reminiscent of ORMs and “you can switch your database and only change the config!” Switching your database, just like switching your cloud provider, rarely happens in practice.
- danryan 3y ago
  That feature was always a byline at best.

Kubernetes means you don't have to learn each clouds way of doing a deployment. You just learn the k8s way then use that with Google, Azure or whatever. So your skillset is reusable.

finikytou 3y ago

no it doesnt mean that. you still need to know how to operate k8s on a specific cloud.

quickthrower2 3y ago
hell yes.
jml78 3y ago
Just like you have to know how to operate each cloud in general. There is no free lunch. But if you learn k8s, moving from AWS EKS to Google GKE to DigitalOceans hosted k8s is easy.
Art9681 3y ago
No you don't have to. You can deploy your own cluster instead of using the managed option if you want to. A good SRE can deploy and manage EKS. A great SRE can deploy and manage a cluster to any Cloud without ever touching the dashboards.
hhh 3y ago
Ideally the developer doesn’t. At scale some platform or infra team should.
artdigital 3y ago
I run stuff on hosted k8s from DO, Google and Vultr. I can absolutely reuse my knowledge, and deployments are almost identical (minus smaller differences like storage csi driver, etc)
- oceanplexian 3y ago
  I work at a place running a million containers deployed in all 3 (Azure, AWS, GCP). I can assure you they are radically different; autoscaling works differently, the load balancers work differently, the networking infrastructure is completely different, the failure modes and limits behaviors are different, the instances perform differently, observability is different, and they all suck in unique and different ways that we discover on a daily basis. Shit even AWS can't keep their regions consistent; each region has different products and features and they fail in different ways. If you are the one maintaining it it's a full time job handling all these edge cases, it's completely miserable and I wouldn't recommend it to anyone.
  - Infernal 3y ago
    > I work at a place running a million containers deployed in all 3 (Azure, AWS, GCP) Are you using AKS, EKS, GKE on those providers, or deploying your own k8s on top of the compute those providers offer? It sounds to me like the former.
    - hosh 3y ago
      I’ve done smaller deployments on GKE and another on EKS, and I can tell you, they are different enough. It’s when you start having to autoscale, optimize resources by instance types, and manage network ingress that these quirks start really come out. The essential ideas are invariant across cloud providers though. But I enjoy working with Kubernetes.
      - Infernal 3y ago
        I should’ve been clearer about what I was getting at. I agree AKS, EKS, GKE etc (cloud gnostic k8s) are different enough to cause a growth of complexity when managing a mixed environment of them. The post I was replying to seemed to be saying (by analogy) “Linux is hard to manage because I run into all sorts of trouble trying to support a mixed environment of SuSE, Ubuntu and RHEL, therefore Linux is just too complicated”.
    - 3y ago
okamiueru 3y ago
Some part of it creates cloud specific resources, and you might also for good reasons have cloud manged database or data storage that your k8s services use. However, "k8s on a specific cloud", is mostly the same, except for the outer edges.
- hosh 3y ago
  Until you want to scale and optimize resources. I enjoy working with Kubernetes, but forcing a complex domain into something legible is a recipe for catastrophe. There are quirks, across cloud providers, and this is just another day in Ops, with or without Kubernetes. (See: https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-called-legibility/ https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-call... )
api 3y ago
It’s more like learning different Linux distributions than learning different OSes.

jcims 3y ago

Kubernetes gets a lot of shit on HN but for all of its challenges it has proven to be a fantastic method to abstract many of the idiosyncrasies of hosting on-prem vs various cloud providers. I've worked for two companies now with 8 figure monthly cloud spend and hundreds/thousands of applications operating in one or more of the main cloud providers and k8s has been essential in making that happen. Teams can migrate to an on-premise hosted option if they want, then transition to cloud if/when it makes sense, or just stay where they are.

stavros 3y ago
Takes notes "Kubernetes has been essential in making 8-figure monthly cloud spend happen"
- windexh8er 3y ago
  The last startup I was in was obsessed with putting everything into k8s for no apparent reason. Even the product they were selling, which most customers hated because it forced our customer base to either have to deal with the pain of paying for k8s because they had no need or intention to use it other than for our product or work with a cross-functional team which now created a time sink and dependency that wouldn't have been there otherwise. The best though was when I ran across someone in the org trying to run a single container to run a periodic job in its own cluster. They spent half the day trying to get it to work with ingress. You can imagine how it came to a head when the company realized they were spending hundreds of thousands per month on idle clusters in AWS.
  - immibis 3y ago
    One company I worked at was obsessed with k8s for a while, on a local arrangement of about 4 servers, each build would start up a new container on kubernetes and rebuild an entire operating system from scratch.
    - immibis 3y ago
      clarification: the thing being built was an operating system. It didn't build an operating system to run the real build on.

oceanplexian 3y ago

> Kubernetes means you don't have to learn each clouds way of doing a deployment. So instead of learning how to deploy on GCP, AWS, and Azure, which is only 3x more complicated than deploying to a single cloud, you should learn K8s, which is 10-15x more complicated, in addition to still having to learn about all the various ingress controllers and weird quirks that are completely different on each cloud provider. Doesn't really track for me.

echelon 3y ago
> which is 10-15x more complicated You can learn k8s in a day. It's really simple. > various ingress controllers and weird quirks that are completely different on each cloud provider Which are thoroughly documented and not that hard to implement or understand. You'd be reading about each cloud's nonstandard ingress even without k8s. The beauty of k8s is you can run your software locally and have a much easier time lifting and shifting to another cloud. Fitting to the shape of a cloud provider is a great way to never leave. Another benefit of k8s is that you treat your services as cattle you can easily spawn and kill. Adoption of k8s naturally leads to anti-fragility, anti-brittle best practices.
- hosh 3y ago
  I think being able to abstract the cloud providers is secondary to k8s’s ability to self-heal and its modularity. But I think most developers don’t care, and instead, should interact with a platform built with Kubernetes as a foundation.
- profunctor 3y ago
  Maybe this is because I’m not that smart but I could not learn real kubernetes in a day. I had to build a system for loading models and returning predictions over a HTTPS api. It had to connect to storage to load the model, needed secrets etc. It took more than a day. And I think it would take most people more than a day to go from zero to able to create a useful, real world deployment in a day. I’m sure you can rush through the documentation in a day but I wouldn’t call that learning.
- bg24 3y ago

berkle4455 3y ago

This argument is reminiscent of ORMs and “you can switch your database and only change the config!” Switching your database, just like switching your cloud provider, rarely happens in practice.

danryan 3y ago
That feature was always a byline at best.