5 ms·

I think several others have commented on this, but the learning curve of Swarm seems insignificant to the setup and implementation of a k8s cluster. I work wit

by bankss 9y ago

I think several others have commented on this, but the learning curve of Swarm seems insignificant to the setup and implementation of a k8s cluster. I work with a team of four total developers and realistically two of us handle the vast majority of "operations." As a result, what was important for us in orchestration was ease of setup, speed of initial implementation, and the lowest immediate and ongoing difficulty associated with whichever orchestration tooling we chose.

pacala 9y ago
I've been running solo a GKE cluster for almost 2 years. Maaaybe I spent 1 week on eng effort on it.
- bankss 9y ago
  GKE, from the research I have done, and the small cluster I built for a side project, smooths out many of the particularly challenging aspects of a k8s implementation. I mean it is advertised as a managed service, I do not think that is particularly comparable to setting up a raw k8s cluster or Docker Swarm cluster yourself. Would you feel that is an accurate assessment? My company is currently fairly locked into another DO for misc. reasons, and as a result would be deploying to DO, where you do not have the benefit of all of the automated tooling/management provided by GKE.
  - pacala 9y ago
    Absolutely. Running k8s on bare metal requires full-time dedicated staff. Running k8s with GKE, and possibly also Azure Container Service, though I have no direct experience, is a one-off half-day effort. What people usually don't realize is that once you're in the cloud, all your services talk TCP. A service in GKE and another one in AWS are just a network hop away. Two considerations are: * Network egress costs, which is currently outrageously high, and serves as a lock-in device of sorts. Depending on actual workloads, it may or maynot make sense for you. * Security. Though there are numerous VPC solutions out there, some of them supported by the cloud providers themselves. Anecdotically, we also run a small RDS database in AWS, though we only need ~100 small queries a day.
    - bankss 9y ago
      Yeah, thanks for sharing your experiences on that. Unfortunately our egress costs would likely be significant. I am honestly anxious to get my employer started on the path to k8s, but until the tooling reduces the hours required to maintain it successfully, Swarm seems like the superior solution if you are locked in on a non-GKE/Azure Container Service provider. I have heard good things about Kops helping setup/enable production-stable orchestration, but they are not ready yet for Digital Ocean either, although I think it is on the list™.
      - hosh 9y ago
        Digital Ocean recently announced a partnering with a Kubernetes vendor. You can run managed K8S on DO now.
        holmb 9y ago
        Do you have a URL for that announcement?
      - jacques_chester 9y ago
        Pivotal and Google built Kubo (recently renamed to the less memorable Cloudfoundry Container Runtime[0]) to make the deployment/management/upgrade thing easier. It's built on top of BOSH, which has a relatively long track record in managing stateful, long-lived, distributed systems on top of IaaSes. Disclosure: I work for Pivotal. [0] https://cloudfoundry.org/container-runtime/ https://cloudfoundry.org/container-runtime/
  - hosh 9y ago
    I have built a prod K8S cluster by hand (back in the 1.1 days). It taught me a lot of the foundations of K8S, so I don't regret it. It is why we are using GKE. With GKE, most of your focus will be around (1) tooling for generating manifests, such as Helm or (forgot the name of it). I had written something called Matsuri, but that is only useful if your team is a Ruby shop. (2) what to put into the containers and how they link up. So yeah, I agree with your assessment.

by bankss 9y ago

pacala 9y ago

I've been running solo a GKE cluster for almost 2 years. Maaaybe I spent 1 week on eng effort on it.

bankss 9y ago
GKE, from the research I have done, and the small cluster I built for a side project, smooths out many of the particularly challenging aspects of a k8s implementation. I mean it is advertised as a managed service, I do not think that is particularly comparable to setting up a raw k8s cluster or Docker Swarm cluster yourself. Would you feel that is an accurate assessment? My company is currently fairly locked into another DO for misc. reasons, and as a result would be deploying to DO, where you do not have the benefit of all of the automated tooling/management provided by GKE.
- pacala 9y ago
  Absolutely. Running k8s on bare metal requires full-time dedicated staff. Running k8s with GKE, and possibly also Azure Container Service, though I have no direct experience, is a one-off half-day effort. What people usually don't realize is that once you're in the cloud, all your services talk TCP. A service in GKE and another one in AWS are just a network hop away. Two considerations are: * Network egress costs, which is currently outrageously high, and serves as a lock-in device of sorts. Depending on actual workloads, it may or maynot make sense for you. * Security. Though there are numerous VPC solutions out there, some of them supported by the cloud providers themselves. Anecdotically, we also run a small RDS database in AWS, though we only need ~100 small queries a day.
  - bankss 9y ago
    Yeah, thanks for sharing your experiences on that. Unfortunately our egress costs would likely be significant. I am honestly anxious to get my employer started on the path to k8s, but until the tooling reduces the hours required to maintain it successfully, Swarm seems like the superior solution if you are locked in on a non-GKE/Azure Container Service provider. I have heard good things about Kops helping setup/enable production-stable orchestration, but they are not ready yet for Digital Ocean either, although I think it is on the list™.
    - hosh 9y ago
      Digital Ocean recently announced a partnering with a Kubernetes vendor. You can run managed K8S on DO now.
      - holmb 9y ago
        Do you have a URL for that announcement?
    - jacques_chester 9y ago
      Pivotal and Google built Kubo (recently renamed to the less memorable Cloudfoundry Container Runtime[0]) to make the deployment/management/upgrade thing easier. It's built on top of BOSH, which has a relatively long track record in managing stateful, long-lived, distributed systems on top of IaaSes. Disclosure: I work for Pivotal. [0] https://cloudfoundry.org/container-runtime/ https://cloudfoundry.org/container-runtime/
- hosh 9y ago
  I have built a prod K8S cluster by hand (back in the 1.1 days). It taught me a lot of the foundations of K8S, so I don't regret it. It is why we are using GKE. With GKE, most of your focus will be around (1) tooling for generating manifests, such as Helm or (forgot the name of it). I had written something called Matsuri, but that is only useful if your team is a Ruby shop. (2) what to put into the containers and how they link up. So yeah, I agree with your assessment.