5 ms·

I have experience running production stacks on Swarm, Kubernetes and ECS. I think the best way to talk about running Swarm stacks in production is to compare it

by marcc 9y ago

I have experience running production stacks on Swarm, Kubernetes and ECS. I think the best way to talk about running Swarm stacks in production is to compare it to other orchestration runtimes. Swarm has a really low learning curve. You can simply write a docker-compose.yml that defines your stack and then run it. The syntax of a docker-compose yaml is likely familiar to many developers who have been using Docker, so that is a huge advantage when adopting. It's a version 3.3 YAML, not the standard docker-compose, but the changes are pretty minor. Swarm stacks come with a built-in overlay (software-defined) network. Services can be identified using in-cluster DNS that round-robins to all containers running that service. This mostly works well, but it's been a challenge to depend on at scale. When a service is rescheduled due to failure, we've seen some instances where requests are dropped. Swarm services are just less specialized than k8s ones, which is the tradeoff for a more approachable, easy-to-use runtime. There's no deployment, service, load balancer, etc. You just run services that depend on other services. Swarm's "depends_on" syntax is simpler, but less robust than what Kubernetes offers. Swarm's secrets are amazing, and really great, but you have to build for them. The idea is that a Swarm Secret doesn't just exist in an unencrypted environment variable, but it's a secured file on disk in the container. The team at Docker did a great job here. While I'm a huge fan of Kubernetes and continuing to push Kubernetes adoption for some of our services, we also use a decent amount of Swarm and plan to continue. Finally, I do worry a little about the Swarm scheduler's future. If anyone from Docker is around, it would be great to hear from them on this. Since they are pushing Kubernetes also, it does make me hesitate to recommend Swarm for a new service, with the risk that it will be end-of-lifed soon.

BretFisher 9y ago
Swarm isn't going away, and in fact, Docker has stated it publicly multiple times on their blog since K8s announcement, and has continued to add new features and fix bugs. Their Kubernetes integrations in Docker EE takes advantage of Swarm's built-in security for node setup. Lots of improvement in the last 6-9 months on Overlay and zero-downtime updates, but true, compared to the first year of SwarmKit, this last 6 months has slowed down in terms of PR's for new features, likely due to focus on K8s integration. Your rolling update issues were likely related to bugs that are (hopefully) now fixed.
- marcc 9y ago
  I agree that most of the issues we encountered were early bugs that have been resolved. While Docker 1.12 and 1.13 had some stability issues with services, it was early and they’ve been addressed. Swarm has been really stable for us since then. I’ve written and shipped a container scheduler/orchestration runtime that’s being used today for some enterprise workloads. It’s a hard problem, and Swarm, Kubernetes and nomad are are solving it well. Swarm is often overlooked as a production grade platform, but it’s absolutely production grade.
- jonballant 9y ago
  "It does make me hesitate to recommend Swarm for a new service, with the risk that it will be end-of-lifed soon." This was my exact concern. With the announcement at the end of 2017 that Docker EE would be integrating Kubernetes, it gave me pause as to whether to put Swarm into production now, when there is the possibility Docker could be moving away from Swarm entirely in the near future. Do you have any sources that there will be continued investment in Swarm? All recent blog posts I see are hyping of the features of the Kubernetes integration.
  - BretFisher 8y ago
    Nov 16h, 2017 Docker creates a blog post covering why Swarm is key to the future of Docker EE and K8s integration: https://blog.docker.com/2017/11/swarm-orchestration-in-docker-enterprise-edition/ https://blog.docker.com/2017/11/swarm-orchestration-in-docke... March 9th, 2018 Docker creates a blog post highlighting a major new feature coming to Swarm in Docker EE 2.0 (alongside K8s integration). https://blog.docker.com/2018/03/enhanced-layer-7-routing-swarm-docker-enterprise-edition-beta/ https://blog.docker.com/2018/03/enhanced-layer-7-routing-swa... There's a lot more evidence that I hope to put into a blog post soon. No company will guarantee they will always make a product, but we've got years of enterprise support for Swarm as it is, and they keep adding functionality so I have no evidence of them stopping or even hinting at such a thing.
  - BretFisher 8y ago
    The TL;DR [direct quote from Docker blog](https://blog.docker.com/2017/11/swarm-orchestration-in-docker-enterprise-edition/ https://blog.docker.com/2017/11/swarm-orchestration-in-docke...) after the Kubernetes announcement: > But it’s equally important for us to note that Swarm orchestration is not going away. Swarm forms an integral cluster management component of the Docker EE platform; in addition, Swarm will operate side-by-side with Kubernetes in a Docker EE cluster, allowing customers to select, based on their needs, the most suitable orchestration tool at application deployment time.

by marcc 9y ago

BretFisher 9y ago

Swarm isn't going away, and in fact, Docker has stated it publicly multiple times on their blog since K8s announcement, and has continued to add new features and fix bugs. Their Kubernetes integrations in Docker EE takes advantage of Swarm's built-in security for node setup. Lots of improvement in the last 6-9 months on Overlay and zero-downtime updates, but true, compared to the first year of SwarmKit, this last 6 months has slowed down in terms of PR's for new features, likely due to focus on K8s integration. Your rolling update issues were likely related to bugs that are (hopefully) now fixed.

marcc 9y ago
I agree that most of the issues we encountered were early bugs that have been resolved. While Docker 1.12 and 1.13 had some stability issues with services, it was early and they’ve been addressed. Swarm has been really stable for us since then. I’ve written and shipped a container scheduler/orchestration runtime that’s being used today for some enterprise workloads. It’s a hard problem, and Swarm, Kubernetes and nomad are are solving it well. Swarm is often overlooked as a production grade platform, but it’s absolutely production grade.
jonballant 9y ago
"It does make me hesitate to recommend Swarm for a new service, with the risk that it will be end-of-lifed soon." This was my exact concern. With the announcement at the end of 2017 that Docker EE would be integrating Kubernetes, it gave me pause as to whether to put Swarm into production now, when there is the possibility Docker could be moving away from Swarm entirely in the near future. Do you have any sources that there will be continued investment in Swarm? All recent blog posts I see are hyping of the features of the Kubernetes integration.
- BretFisher 8y ago
  Nov 16h, 2017 Docker creates a blog post covering why Swarm is key to the future of Docker EE and K8s integration: https://blog.docker.com/2017/11/swarm-orchestration-in-docker-enterprise-edition/ https://blog.docker.com/2017/11/swarm-orchestration-in-docke... March 9th, 2018 Docker creates a blog post highlighting a major new feature coming to Swarm in Docker EE 2.0 (alongside K8s integration). https://blog.docker.com/2018/03/enhanced-layer-7-routing-swarm-docker-enterprise-edition-beta/ https://blog.docker.com/2018/03/enhanced-layer-7-routing-swa... There's a lot more evidence that I hope to put into a blog post soon. No company will guarantee they will always make a product, but we've got years of enterprise support for Swarm as it is, and they keep adding functionality so I have no evidence of them stopping or even hinting at such a thing.
- BretFisher 8y ago
  The TL;DR [direct quote from Docker blog](https://blog.docker.com/2017/11/swarm-orchestration-in-docker-enterprise-edition/ https://blog.docker.com/2017/11/swarm-orchestration-in-docke...) after the Kubernetes announcement: > But it’s equally important for us to note that Swarm orchestration is not going away. Swarm forms an integral cluster management component of the Docker EE platform; in addition, Swarm will operate side-by-side with Kubernetes in a Docker EE cluster, allowing customers to select, based on their needs, the most suitable orchestration tool at application deployment time.