9 ms·

What if you're on a hosted service like gke/aks? The work needed to maintain the system has been minimal for us with a lot of positives

bauerd 8y ago
Yeah +1. I was super skeptical 1,5 months ago when I started my new job on a 22 node GKE cluster. "Surely K8s is overkill", "could've just used Nomad + Consul + Vault + X", "why not just use Fargate", "K8s seems like a distributed systemd", etc pp. Long story short, K8s on GKE just works, saves us tons of time, makes service-oriented architectures/microservices easy as pie and has led to a way better service level than the previous setup (Chef deployed on baremetal). Yes, it probably is overkill for your 10req/s app, granted. But I highly advise everyone to at least give it a try out of curiosity, because a lot of hard problems at medium scale and above just go poof with K8s.
- firebacon 8y ago
  > because a lot of hard problems at medium scale and above just go poof with K8s. No, they don't? I don't know why anybody would just assume that something as complex as kubernetes would just run flawlessly once you actually try to run it on thousands of servers. Must be something to do with google PR because people definitely don't seem to assume the same for e.g. hadoop or openstack. Make a guess at how many people large companies have to employ to actually keep their smart cluster scheduler running?
  - bauerd 8y ago
    I was commenting on a specific usage context: >when I started my new job on a 22 node GKE cluster >problems at medium scale vs >Make a guess at how many people large companies have to employ to actually keep their smart cluster scheduler running K8s obviously is not a silver bullet and of course there's ops work to be done. I can't comment on whether operating K8s clusters in other contexts makes economic sense, but I know it does for our current team.
    - firebacon 8y ago
      > makes economic sense [...] for our current team. But the reason for that is not that it makes "hard problems go poof at scale". The reason is that you're using a hosted service where somebody else (in this case, Google themselves) takes care of the problems for you for a fee and - at small scale - you only have to pay them a fraction of a single operation's engineers salary for it. So of course it makes economic sense for you to use a hosted service where the sharing economy kicks in, but your recommendation to use kubernetes because it solves hard technical problems at medium scale does not follow from that.
      - jjeaff 8y ago
        If you are running on Google computer anyway, then I don't think you pay anything extra for gke. As far as I can tell, you are only paying for the VMs that you use. So the price for hosted k8s is already baked in whether you use it or not.
      - dominotw 8y ago
        It was a response to what if gke question. So it follows in that context .
        firebacon 8y ago
        No it doesn't. You can not extrapolate from the fact that the hosted version "just works" that kubernetes at scale would also "just work" (kubernetes being the open source product that you run yourself here). Especially if the hosted version is offered by google and everybody knows that they employ truckloads of very good engineers keeping their stuff running.
        brazzledazzle 8y ago
        The original comment asked “what about hosted kubernetes?” and the reply was discussing the benefits of that hosted solution. They didn’t say that running your own cluster would be the same. They didn’t even imply it. Even if they had, it doesn’t justify your aggression. Maybe take a break from the internet for a bit to clear your head.
akhilcacharya 8y ago
Would it be accurate to describe some of the K8S ecosystem as building out equivalent services (monitoring, failover, health checks) to the cloud providers on top of a infra-agnostic layer?
- pests 8y ago
  This is how I use it. My applications can now be cloud-agnostic besides any other services I use with that provider.
claytonjy 8y ago
I'm another happy user of this approach. While I generally agree with other posters all over these comments saying it isn't worth it for small or even medium deployments, I'm happy using it with a team of 2-3 for a single client. Much different use-case than most, though: automated machine learning pipelines via Pachyderm [1]. Would never have touched K8's if it wasn't for Pachd, and between their layer and GKE it's been quite painless. May have given up on Pachyderm if it wasn't for GKE; we're not cut out for maintaining K8's, and our first shot on AWS (pre-AKS) was not pretty. tl;dr: K8's are great when it's all but completely abstracted away from you [1] http://www.pachyderm.io/ http://www.pachyderm.io/

bauerd 8y ago

Yeah +1. I was super skeptical 1,5 months ago when I started my new job on a 22 node GKE cluster. "Surely K8s is overkill", "could've just used Nomad + Consul + Vault + X", "why not just use Fargate", "K8s seems like a distributed systemd", etc pp. Long story short, K8s on GKE just works, saves us tons of time, makes service-oriented architectures/microservices easy as pie and has led to a way better service level than the previous setup (Chef deployed on baremetal). Yes, it probably is overkill for your 10req/s app, granted. But I highly advise everyone to at least give it a try out of curiosity, because a lot of hard problems at medium scale and above just go poof with K8s.

firebacon 8y ago
> because a lot of hard problems at medium scale and above just go poof with K8s. No, they don't? I don't know why anybody would just assume that something as complex as kubernetes would just run flawlessly once you actually try to run it on thousands of servers. Must be something to do with google PR because people definitely don't seem to assume the same for e.g. hadoop or openstack. Make a guess at how many people large companies have to employ to actually keep their smart cluster scheduler running?
- bauerd 8y ago
  I was commenting on a specific usage context: >when I started my new job on a 22 node GKE cluster >problems at medium scale vs >Make a guess at how many people large companies have to employ to actually keep their smart cluster scheduler running K8s obviously is not a silver bullet and of course there's ops work to be done. I can't comment on whether operating K8s clusters in other contexts makes economic sense, but I know it does for our current team.
  - firebacon 8y ago
    > makes economic sense [...] for our current team. But the reason for that is not that it makes "hard problems go poof at scale". The reason is that you're using a hosted service where somebody else (in this case, Google themselves) takes care of the problems for you for a fee and - at small scale - you only have to pay them a fraction of a single operation's engineers salary for it. So of course it makes economic sense for you to use a hosted service where the sharing economy kicks in, but your recommendation to use kubernetes because it solves hard technical problems at medium scale does not follow from that.
    - jjeaff 8y ago
      If you are running on Google computer anyway, then I don't think you pay anything extra for gke. As far as I can tell, you are only paying for the VMs that you use. So the price for hosted k8s is already baked in whether you use it or not.
    - dominotw 8y ago
      It was a response to what if gke question. So it follows in that context .
      - firebacon 8y ago
        No it doesn't. You can not extrapolate from the fact that the hosted version "just works" that kubernetes at scale would also "just work" (kubernetes being the open source product that you run yourself here). Especially if the hosted version is offered by google and everybody knows that they employ truckloads of very good engineers keeping their stuff running.
        brazzledazzle 8y ago
        The original comment asked “what about hosted kubernetes?” and the reply was discussing the benefits of that hosted solution. They didn’t say that running your own cluster would be the same. They didn’t even imply it. Even if they had, it doesn’t justify your aggression. Maybe take a break from the internet for a bit to clear your head.

akhilcacharya 8y ago

Would it be accurate to describe some of the K8S ecosystem as building out equivalent services (monitoring, failover, health checks) to the cloud providers on top of a infra-agnostic layer?

pests 8y ago
This is how I use it. My applications can now be cloud-agnostic besides any other services I use with that provider.

claytonjy 8y ago

I'm another happy user of this approach. While I generally agree with other posters all over these comments saying it isn't worth it for small or even medium deployments, I'm happy using it with a team of 2-3 for a single client. Much different use-case than most, though: automated machine learning pipelines via Pachyderm [1]. Would never have touched K8's if it wasn't for Pachd, and between their layer and GKE it's been quite painless. May have given up on Pachyderm if it wasn't for GKE; we're not cut out for maintaining K8's, and our first shot on AWS (pre-AKS) was not pretty. tl;dr: K8's are great when it's all but completely abstracted away from you [1] http://www.pachyderm.io/ http://www.pachyderm.io/