9 ms·

Why did you pick Swarm for production? We followed Swarm from the beginning, but after a few releases at v0.4 it was clear not to ever use Swarm, and that it m

Why did you pick Swarm for production? We followed Swarm from the beginning, but after a few releases at v0.4 it was clear not to ever use Swarm, and that it mostly was the Docker PR machine that made it sound nice, and not the actual features. Maybe it got better later on, but the first several Swarm announcements seemed really off-putting to me. We ended up on Mesos/Marathon, not that that has a bright future either, but it was at least capable of restarting containers from the beginning.. Just migrate to Kubernetes. It has won.

eecc 9y ago
Hmm, that k8s “has won” might make me a bit sad - not that I’ve ever gasped some fresh air out of the stranglehold of AWS - as I was impressed with Mesos. Can you share some 1st person opinions on Mesos, and where K8s is a step forward, backward or aside? TIA
- tveita 9y ago
  Having moved from Mesos to Kubernetes, Kubernetes just felt more mature. Working solutions for stateful sets, service discovery with DNS, flexible scheduling with affinity and tolerance, saner resource limits, a good CLI tool. It's not a completely fair comparison since we also were able to offload persistent storage to Google Cloud, which is one of the harder problems IMO. I think Mesos has improved since then, but it always felt like they were a bit behind. In general Kubernetes feels like it is designed by people with relevant experience. Especially compared to our earlier experiments with Docker Compose files. People are praising their simplicity, but they left us solving a lot of hard problems that Kubernetes solves for us better than we could have done.
- manigandham 9y ago
  Why is it sad? It's great that we can finally standardize and use a single powerful system that is very capable but also improving quickly.
smoyer 9y ago
When we started looking for a container orchestration system, we naturally used Google Trends (https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes https://trends.google.com/trends/explore?date=today%205-y&q=...). Kubernetes reached their 1.0 release shortly before we were ready to start using the system and a lot of the features added since then eliminated so of our other problems (e.g. configmaps/secrets combined with the existing service discovery pretty much eliminated our plans to use Consul). I've been taking a second look at Mesos recently and found that I didn't really grok it the first time I looked at it. In any case, I think your assessment is correct ("Just migrate to Kubernetes" - https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes,mesos https://trends.google.com/trends/explore?date=today%205-y&q=...).
sz4kerto 9y ago
The actual features are very nice. I don't know what else makes you say 'migrate' apart from the fact that there's a growing support for k8s. The advantages of Swarm (very easy setup in private clouds, docker-compose format descriptors,etc.) don't go away just because k8s is popular. Stop spreading FUD please, it is not good for anyone.
sandGorgon 9y ago
i would encourage you to try Swarm. Its brilliant in its simplicity. I think Docker product marketing and customer success basically sucks. But Swarm as a product has been really, really nice. And yes I continuously evaluate kubernetes and swarm side by side. You can get a swarm cluster running in less than 10 minutes on your local laptop after "apt-get install docker-ce". To run k8s, you will need to first muck about ingress and overlays and everything else. I know its because of "flexibility" - its like Sinatra vs Rails. They are both great in their spaces.
- pacala 9y ago
  $ brew cask install minikube $ minikube start
  - sandGorgon 9y ago
    minikube is a specific version/distro/packaging of kubernetes meant for testing on local laptop. Docker Swarm runs exactly the same way with exactly the same components and with the same ease on laptop as well as the cloud. TL;DR - you cant run minikube in production.
    - orf 9y ago
      It's just Kubernetes in a VM, like Docker on Mac is just docker in a VM. You can run Kubernetes on your Linux device machine as-is.
      - Axsuul 9y ago
        It's what's in the VM that counts.
    - wstrange 9y ago
      Creating a k8s cluster on GKE is just as easy (a single gcloud command, or use the GUI if that is your preference). With hosted Kubernetes as a service (GKE, AKS and soon EKS), there is little reason to roll your own cluster.
jrs95 9y ago
Is HashiCorp’s Nomad in a similar position at this point? I really enjoy it’s (relative) simplicity.
- hosh 9y ago
  I don't know about Nomad, though Hashicorp's Vault is going to do just fine I think. It fills in a gap in secrets management that K8S doesn't do out of the box. Looking at doing Vault in HA leads people to look at Consul, which leads to Nomad. (Consul uses a consensus protocol for service discovery and I think that will be interesting for the next generation). Last year, K8S had already captured the center of gravity, and it took a while for the rest of the dev community to catch up. I think this year ia a lot of shuffling as the survivors settles into orbit around K8S. There is a lot of interesting innovations up the stack once orchestration is de facto standardized. K8S still hasn't solved the stateful workloads, though it is introducing a lot of primitives to support those: controller hooks, third part resources, on top of which Operators can function. I think we will see a lot more innovations as people create Operators. That can include anything from stateful loads for specific distributed stateful loads, to things like intrusion detection, ML-driven autoscaling, and so forth.
- wmf 9y ago
  I get the impression that Nomad was never particularly alive to begin with, which is a shame since it seems better designed. But it doesn't have that "ZOMG Google has blessed us with the secrets of the borg" that DevOps crave.
  - rmetzler 9y ago
    Nomad was my choice for queue centric workloads, but it doesn't seem to fit the webserver / long living services as good as Kubernetes. I'm not sure, but I would think you could run Nomad and Kubernetes on the same servers, sharing the Docker runtime.
    - wmf 9y ago
      If you run two schedulers they don't have a correct view of available capacity. You can use Mesos as a meta-scheduler but that introduces more complexity.
dahidahi1 9y ago
There were two swarm's, the swarm classic — which was okayy, & then the newer swarm Mode, introduced in docker 1.12. I'm not sure which one you refer to when you say v0.4. But swarm mode was good, extremely simple & worked well for the right workloads. Like most solutions it is not a silver bullet for every orchestration need, but worked very well in microservices, new-ish architectures. K8s is great too, but it seems like an overkill for a handful of services. Also setup of K8s used to be hard, especially HA. The learning curve is also quite steep. One of the features in swarm mode that makes docker swarm extremely intuitive — IPVS, is being incorporated in K8s. So I guess there has been some cross pollination on both sides. But I do not think the swarm mode is going to die anytime soon.

eecc 9y ago

Hmm, that k8s “has won” might make me a bit sad - not that I’ve ever gasped some fresh air out of the stranglehold of AWS - as I was impressed with Mesos. Can you share some 1st person opinions on Mesos, and where K8s is a step forward, backward or aside? TIA

tveita 9y ago
Having moved from Mesos to Kubernetes, Kubernetes just felt more mature. Working solutions for stateful sets, service discovery with DNS, flexible scheduling with affinity and tolerance, saner resource limits, a good CLI tool. It's not a completely fair comparison since we also were able to offload persistent storage to Google Cloud, which is one of the harder problems IMO. I think Mesos has improved since then, but it always felt like they were a bit behind. In general Kubernetes feels like it is designed by people with relevant experience. Especially compared to our earlier experiments with Docker Compose files. People are praising their simplicity, but they left us solving a lot of hard problems that Kubernetes solves for us better than we could have done.
manigandham 9y ago
Why is it sad? It's great that we can finally standardize and use a single powerful system that is very capable but also improving quickly.

smoyer 9y ago

When we started looking for a container orchestration system, we naturally used Google Trends (https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes https://trends.google.com/trends/explore?date=today%205-y&q=...). Kubernetes reached their 1.0 release shortly before we were ready to start using the system and a lot of the features added since then eliminated so of our other problems (e.g. configmaps/secrets combined with the existing service discovery pretty much eliminated our plans to use Consul). I've been taking a second look at Mesos recently and found that I didn't really grok it the first time I looked at it. In any case, I think your assessment is correct ("Just migrate to Kubernetes" - https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes,mesos https://trends.google.com/trends/explore?date=today%205-y&q=...).

sz4kerto 9y ago

The actual features are very nice. I don't know what else makes you say 'migrate' apart from the fact that there's a growing support for k8s. The advantages of Swarm (very easy setup in private clouds, docker-compose format descriptors,etc.) don't go away just because k8s is popular. Stop spreading FUD please, it is not good for anyone.

sandGorgon 9y ago

i would encourage you to try Swarm. Its brilliant in its simplicity. I think Docker product marketing and customer success basically sucks. But Swarm as a product has been really, really nice. And yes I continuously evaluate kubernetes and swarm side by side. You can get a swarm cluster running in less than 10 minutes on your local laptop after "apt-get install docker-ce". To run k8s, you will need to first muck about ingress and overlays and everything else. I know its because of "flexibility" - its like Sinatra vs Rails. They are both great in their spaces.

pacala 9y ago
$ brew cask install minikube $ minikube start
- sandGorgon 9y ago
  minikube is a specific version/distro/packaging of kubernetes meant for testing on local laptop. Docker Swarm runs exactly the same way with exactly the same components and with the same ease on laptop as well as the cloud. TL;DR - you cant run minikube in production.
  - orf 9y ago
    It's just Kubernetes in a VM, like Docker on Mac is just docker in a VM. You can run Kubernetes on your Linux device machine as-is.
    - Axsuul 9y ago
      It's what's in the VM that counts.
  - wstrange 9y ago
    Creating a k8s cluster on GKE is just as easy (a single gcloud command, or use the GUI if that is your preference). With hosted Kubernetes as a service (GKE, AKS and soon EKS), there is little reason to roll your own cluster.

jrs95 9y ago

Is HashiCorp’s Nomad in a similar position at this point? I really enjoy it’s (relative) simplicity.

hosh 9y ago
I don't know about Nomad, though Hashicorp's Vault is going to do just fine I think. It fills in a gap in secrets management that K8S doesn't do out of the box. Looking at doing Vault in HA leads people to look at Consul, which leads to Nomad. (Consul uses a consensus protocol for service discovery and I think that will be interesting for the next generation). Last year, K8S had already captured the center of gravity, and it took a while for the rest of the dev community to catch up. I think this year ia a lot of shuffling as the survivors settles into orbit around K8S. There is a lot of interesting innovations up the stack once orchestration is de facto standardized. K8S still hasn't solved the stateful workloads, though it is introducing a lot of primitives to support those: controller hooks, third part resources, on top of which Operators can function. I think we will see a lot more innovations as people create Operators. That can include anything from stateful loads for specific distributed stateful loads, to things like intrusion detection, ML-driven autoscaling, and so forth.
wmf 9y ago
I get the impression that Nomad was never particularly alive to begin with, which is a shame since it seems better designed. But it doesn't have that "ZOMG Google has blessed us with the secrets of the borg" that DevOps crave.
- rmetzler 9y ago
  Nomad was my choice for queue centric workloads, but it doesn't seem to fit the webserver / long living services as good as Kubernetes. I'm not sure, but I would think you could run Nomad and Kubernetes on the same servers, sharing the Docker runtime.
  - wmf 9y ago
    If you run two schedulers they don't have a correct view of available capacity. You can use Mesos as a meta-scheduler but that introduces more complexity.

dahidahi1 9y ago

There were two swarm's, the swarm classic — which was okayy, & then the newer swarm Mode, introduced in docker 1.12. I'm not sure which one you refer to when you say v0.4. But swarm mode was good, extremely simple & worked well for the right workloads. Like most solutions it is not a silver bullet for every orchestration need, but worked very well in microservices, new-ish architectures. K8s is great too, but it seems like an overkill for a handful of services. Also setup of K8s used to be hard, especially HA. The learning curve is also quite steep. One of the features in swarm mode that makes docker swarm extremely intuitive — IPVS, is being incorporated in K8s. So I guess there has been some cross pollination on both sides. But I do not think the swarm mode is going to die anytime soon.