7 ms·

We are extremely worried about the future of Docker Swarm as well. We love Swarm - but we are seeing most work out of the Docker team is to give a migration p

We are extremely worried about the future of Docker Swarm as well. We love Swarm - but we are seeing most work out of the Docker team is to give a migration path to kubernetes. A huge number of docker swarm networking bugs are not being worked on. We will be happy if Docker talks about Swarm becoming a management UX for K8s - but we need visibility. These are production orchestration systems. The migration path is not easy. And seeing what Docker Co is doing with Cloud, it is not very comforting to trust that they will do the right thing with Swarm.

cygned 9y ago
I had to make a decision for an orchestration tool a few weeks ago and I went with K8s. One of the main reasons was that even Docker advertises it on its website and with Docker for Mac. I expect Swarm support to be canceled in a not so distant future and I cannot rely on a tool with an unclear future. Which is a pity because I really liked Swarm for its simplicity. Side note: I am also concerned about Docker in general. CE/EE split, services shutting down, bugs seemingly not being fixed - I cannot point out a precise aspect, but I am concerned.
- ianai 9y ago
  Who’s beating them? Amazon?
  - sytse 9y ago
    Kubernetes had won the container scheduler wars. At GitLab we're all in on making a PaaS based on k8s and our CI/CD and the container registry that is part of GitLab.
    - candiodari 9y ago
      It feels awfully 19th century though that despite k8s having "won", by far the biggest container schedulers by containers scheduled are, no doubt: (I think this is the correct order, not 100% sure of course) 1) google borg (maybe omega) [1] 2) amazon ec2 3) whatever microsoft is using (large gap) 4) all the rest of the world combined, a small portion of which is k8s [1] https://www.quora.com/Does-Google-use-the-Open-Source-Kubernetes-or-a-version-of-Borg-for-their-own-container-management https://www.quora.com/Does-Google-use-the-Open-Source-Kubern... (one might even say [1] seems to imply it'll never happen, or at least take a very long time. Also if you read the papers it becomes very clear that "Google Borg" includes a lot of things these days at many levels, from custom ASICs, device firmware (as in standard device, google borg firmware), BIOS firmware, entirely custom sub-kernel code, custom kernels, custom userspace (ie. Google-specific libc that's not optional), ... all of these will turn out to have dependencies on eachother that have to be redone for k8s, could take a while to migrate over) (although I have not read any papers on it (I'd love some though), I'd bet amazon is in a similar boat, and of course Microsoft is Microsoft)
      - gaius 9y ago
        EC2 is not a container scheduler - it's an IaaS for VMs. The Amazon container PaaS (ECS/EKS) is a layer on top of EC2. And that is being superseded by Fargate which will make the underlying EC2 invisible. If you need a Fargate-like capability now, Azure AKS does it. See https://azure.microsoft.com/en-us/services/container-service/ https://azure.microsoft.com/en-us/services/container-service... and https://aws.amazon.com/fargate/ https://aws.amazon.com/fargate/
        candiodari 9y ago
        So what is the EC2 container scheduler before Fargate called ? Any papers on it ?
        gaius 9y ago
        ECS and EKS.
        coredog64 9y ago
        Fargate is expensive as hell for long running services. You should only be using it for something that creates value 100% of the time that it is running.
    - sandGorgon 9y ago
      > At GitLab we're all in on making a PaaS based on k8s This is very interesting. Could you talk more on this ? There is definitely space for an "opinionated k8s distro with batteries included". I have wished for Swarm to become this....
      - sytse 9y ago
        It is not a Kubernetes distribution. You can use any distribution or CaaS you want. The beginning of it is in GitLab Auto DevOps https://about.gitlab.com/2017/10/04/devops-strategy/ https://about.gitlab.com/2017/10/04/devops-strategy/
    - komuW 9y ago
      Interesting. Is there a blog post where i can read more about this?
      - sytse 9y ago
        https://docs.gitlab.com/ee/topics/autodevops/ https://docs.gitlab.com/ee/topics/autodevops/ and https://about.gitlab.com/2017/10/04/devops-strategy/ https://about.gitlab.com/2017/10/04/devops-strategy/
- cwingrav 9y ago
  I'm not sure who's going to beat Docker. Docker is central to most orchestration tools so as long as they make money someplace with their central services, they should be fine.
  - iamdeedubs 9y ago
    The kubernetes community is pouring a lot of resources into cri-o. I imagine you are going to see the kubernetes clusters that are built 'the hard way' start switching over and removing docker. It will still be used for building pushing containers for the time being.
  - xorcist 9y ago
    Kubernetes could move to rkt or even the now standard systemd stuff, end users would hardly know the difference. The container format isn't a very strong lock-in effect and most people are probably better served without the image type format anyway (as the Linux block layer wasn't really constructed with that use case in mind, and fixing the plumbing will take longer time than developing the orchestration tools which is what'll win the users). Docker the company have few options to monetize on Docker the software when it becomes commoditized. They seeminly chose the Enterprise way, which consists of pretty orchestration tools and integrations with Active Directory. (A perfectly valid option, which worked out well for VMware.) That's a dead end now that Kubernetes won container orchestration. It will be interesting to see where they will go next.
- gortok 9y ago
  I’m concerned as well. We use Docker and Docker Compose heavily for our development and both on Docker for Windows and Docker for Mac developers have to restart their daemon several times a day. The binaries aren’t open so it’s tough to see and fix the issue; but because we aren’t Docker Enterprise Engine customers, there is no path to support. It would be helpful if there were a way to pay for Docker and receive support without having to go the enterprise route. I can see paying $200 a month for the team for support.
  - toomuchtodo 9y ago
    Yikes. Surely Docker is providing you more than $200 worth of value a month. If so, why would you only pay $200?
    - gortok 9y ago
      I might; but that’d have to mean seeing some traction from that money first. They haven’t proven that they’re able to run the sort of business they’re trying to run.
Axsuul 9y ago
I hope this isn't so either. Docker Swarm is so simple and works well for many use cases.
CSDude 9y ago
This is already Swarm V2, there was older Swarm, which worked nice enough. It was equivalent of multi-host Docker run, which could filter based on constraints and do bin-packing, and had even support for multi host networking with etcd/consul/zookeeper. Then, they cancelled it, no more patches, no mention of it anywhere unless you know where to look. Then they created Swarm Mode, and add the concept of "services" which sucks compared to regular run because it lacked so many options the run command had, it took more than 6 months to implement most of them.
adamtulinius 9y ago
Why did you pick Swarm for production? We followed Swarm from the beginning, but after a few releases at v0.4 it was clear not to ever use Swarm, and that it mostly was the Docker PR machine that made it sound nice, and not the actual features. Maybe it got better later on, but the first several Swarm announcements seemed really off-putting to me. We ended up on Mesos/Marathon, not that that has a bright future either, but it was at least capable of restarting containers from the beginning.. Just migrate to Kubernetes. It has won.
- eecc 9y ago
  Hmm, that k8s “has won” might make me a bit sad - not that I’ve ever gasped some fresh air out of the stranglehold of AWS - as I was impressed with Mesos. Can you share some 1st person opinions on Mesos, and where K8s is a step forward, backward or aside? TIA
  - tveita 9y ago
    Having moved from Mesos to Kubernetes, Kubernetes just felt more mature. Working solutions for stateful sets, service discovery with DNS, flexible scheduling with affinity and tolerance, saner resource limits, a good CLI tool. It's not a completely fair comparison since we also were able to offload persistent storage to Google Cloud, which is one of the harder problems IMO. I think Mesos has improved since then, but it always felt like they were a bit behind. In general Kubernetes feels like it is designed by people with relevant experience. Especially compared to our earlier experiments with Docker Compose files. People are praising their simplicity, but they left us solving a lot of hard problems that Kubernetes solves for us better than we could have done.
  - manigandham 9y ago
    Why is it sad? It's great that we can finally standardize and use a single powerful system that is very capable but also improving quickly.
- smoyer 9y ago
  When we started looking for a container orchestration system, we naturally used Google Trends (https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes https://trends.google.com/trends/explore?date=today%205-y&q=...). Kubernetes reached their 1.0 release shortly before we were ready to start using the system and a lot of the features added since then eliminated so of our other problems (e.g. configmaps/secrets combined with the existing service discovery pretty much eliminated our plans to use Consul). I've been taking a second look at Mesos recently and found that I didn't really grok it the first time I looked at it. In any case, I think your assessment is correct ("Just migrate to Kubernetes" - https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes,mesos https://trends.google.com/trends/explore?date=today%205-y&q=...).
robbyt 9y ago
Docker is a amazing tool, but I think the technical design and overall strategy for Swarm wasn't very well executed. Moving to k8s is a smart thing for them, because it's objectively better for real production use. In our tests about a year ago, swarm started showing serious networking and cluster synchronization problems with cluster sizes over 30 nodes (physical servers), on a fast, reliable LAN. I've heard similar stories from another big Docker customer- Docker support promised them that improving performance of Swarm and fixing scaling issues are the focus of "the next version", but they never came. This company is now moving to k8s.
BretFisher 9y ago
Could it be that the teams are simply focused on adding K8s support and getting the Docker EE out of Beta? Public statement on their blog after the K8s announcement in EU: "But it’s equally important for us to note that Swarm orchestration is not going away. Swarm forms an integral cluster management component of the Docker EE platform; in addition, Swarm will operate side-by-side with Kubernetes in a Docker EE cluster, allowing customers to select, based on their needs, the most suitable orchestration tool at application deployment time." https://blog.docker.com/2017/11/swarm-orchestration-in-docker-enterprise-edition/ https://blog.docker.com/2017/11/swarm-orchestration-in-docke... There's still plenty of PR's and activity in the SwarmKit and Libnetwork repos: https://github.com/docker/swarmkit/pulse/monthly https://github.com/docker/swarmkit/pulse/monthly https://github.com/docker/libnetwork/pulse/monthly https://github.com/docker/libnetwork/pulse/monthly

cygned 9y ago

I had to make a decision for an orchestration tool a few weeks ago and I went with K8s. One of the main reasons was that even Docker advertises it on its website and with Docker for Mac. I expect Swarm support to be canceled in a not so distant future and I cannot rely on a tool with an unclear future. Which is a pity because I really liked Swarm for its simplicity. Side note: I am also concerned about Docker in general. CE/EE split, services shutting down, bugs seemingly not being fixed - I cannot point out a precise aspect, but I am concerned.

ianai 9y ago
Who’s beating them? Amazon?
- sytse 9y ago
  Kubernetes had won the container scheduler wars. At GitLab we're all in on making a PaaS based on k8s and our CI/CD and the container registry that is part of GitLab.
  - candiodari 9y ago
    It feels awfully 19th century though that despite k8s having "won", by far the biggest container schedulers by containers scheduled are, no doubt: (I think this is the correct order, not 100% sure of course) 1) google borg (maybe omega) [1] 2) amazon ec2 3) whatever microsoft is using (large gap) 4) all the rest of the world combined, a small portion of which is k8s [1] https://www.quora.com/Does-Google-use-the-Open-Source-Kubernetes-or-a-version-of-Borg-for-their-own-container-management https://www.quora.com/Does-Google-use-the-Open-Source-Kubern... (one might even say [1] seems to imply it'll never happen, or at least take a very long time. Also if you read the papers it becomes very clear that "Google Borg" includes a lot of things these days at many levels, from custom ASICs, device firmware (as in standard device, google borg firmware), BIOS firmware, entirely custom sub-kernel code, custom kernels, custom userspace (ie. Google-specific libc that's not optional), ... all of these will turn out to have dependencies on eachother that have to be redone for k8s, could take a while to migrate over) (although I have not read any papers on it (I'd love some though), I'd bet amazon is in a similar boat, and of course Microsoft is Microsoft)
    - gaius 9y ago
      EC2 is not a container scheduler - it's an IaaS for VMs. The Amazon container PaaS (ECS/EKS) is a layer on top of EC2. And that is being superseded by Fargate which will make the underlying EC2 invisible. If you need a Fargate-like capability now, Azure AKS does it. See https://azure.microsoft.com/en-us/services/container-service/ https://azure.microsoft.com/en-us/services/container-service... and https://aws.amazon.com/fargate/ https://aws.amazon.com/fargate/
      - candiodari 9y ago
        So what is the EC2 container scheduler before Fargate called ? Any papers on it ?
        gaius 9y ago
        ECS and EKS.
      - coredog64 9y ago
        Fargate is expensive as hell for long running services. You should only be using it for something that creates value 100% of the time that it is running.
  - sandGorgon 9y ago
    > At GitLab we're all in on making a PaaS based on k8s This is very interesting. Could you talk more on this ? There is definitely space for an "opinionated k8s distro with batteries included". I have wished for Swarm to become this....
    - sytse 9y ago
      It is not a Kubernetes distribution. You can use any distribution or CaaS you want. The beginning of it is in GitLab Auto DevOps https://about.gitlab.com/2017/10/04/devops-strategy/ https://about.gitlab.com/2017/10/04/devops-strategy/
  - komuW 9y ago
    Interesting. Is there a blog post where i can read more about this?
    - sytse 9y ago
      https://docs.gitlab.com/ee/topics/autodevops/ https://docs.gitlab.com/ee/topics/autodevops/ and https://about.gitlab.com/2017/10/04/devops-strategy/ https://about.gitlab.com/2017/10/04/devops-strategy/
cwingrav 9y ago
I'm not sure who's going to beat Docker. Docker is central to most orchestration tools so as long as they make money someplace with their central services, they should be fine.
- iamdeedubs 9y ago
  The kubernetes community is pouring a lot of resources into cri-o. I imagine you are going to see the kubernetes clusters that are built 'the hard way' start switching over and removing docker. It will still be used for building pushing containers for the time being.
- xorcist 9y ago
  Kubernetes could move to rkt or even the now standard systemd stuff, end users would hardly know the difference. The container format isn't a very strong lock-in effect and most people are probably better served without the image type format anyway (as the Linux block layer wasn't really constructed with that use case in mind, and fixing the plumbing will take longer time than developing the orchestration tools which is what'll win the users). Docker the company have few options to monetize on Docker the software when it becomes commoditized. They seeminly chose the Enterprise way, which consists of pretty orchestration tools and integrations with Active Directory. (A perfectly valid option, which worked out well for VMware.) That's a dead end now that Kubernetes won container orchestration. It will be interesting to see where they will go next.
gortok 9y ago
I’m concerned as well. We use Docker and Docker Compose heavily for our development and both on Docker for Windows and Docker for Mac developers have to restart their daemon several times a day. The binaries aren’t open so it’s tough to see and fix the issue; but because we aren’t Docker Enterprise Engine customers, there is no path to support. It would be helpful if there were a way to pay for Docker and receive support without having to go the enterprise route. I can see paying $200 a month for the team for support.
- toomuchtodo 9y ago
  Yikes. Surely Docker is providing you more than $200 worth of value a month. If so, why would you only pay $200?
  - gortok 9y ago
    I might; but that’d have to mean seeing some traction from that money first. They haven’t proven that they’re able to run the sort of business they’re trying to run.

Axsuul 9y ago

I hope this isn't so either. Docker Swarm is so simple and works well for many use cases.

CSDude 9y ago

This is already Swarm V2, there was older Swarm, which worked nice enough. It was equivalent of multi-host Docker run, which could filter based on constraints and do bin-packing, and had even support for multi host networking with etcd/consul/zookeeper. Then, they cancelled it, no more patches, no mention of it anywhere unless you know where to look. Then they created Swarm Mode, and add the concept of "services" which sucks compared to regular run because it lacked so many options the run command had, it took more than 6 months to implement most of them.

adamtulinius 9y ago

Why did you pick Swarm for production? We followed Swarm from the beginning, but after a few releases at v0.4 it was clear not to ever use Swarm, and that it mostly was the Docker PR machine that made it sound nice, and not the actual features. Maybe it got better later on, but the first several Swarm announcements seemed really off-putting to me. We ended up on Mesos/Marathon, not that that has a bright future either, but it was at least capable of restarting containers from the beginning.. Just migrate to Kubernetes. It has won.

eecc 9y ago
Hmm, that k8s “has won” might make me a bit sad - not that I’ve ever gasped some fresh air out of the stranglehold of AWS - as I was impressed with Mesos. Can you share some 1st person opinions on Mesos, and where K8s is a step forward, backward or aside? TIA
- tveita 9y ago
  Having moved from Mesos to Kubernetes, Kubernetes just felt more mature. Working solutions for stateful sets, service discovery with DNS, flexible scheduling with affinity and tolerance, saner resource limits, a good CLI tool. It's not a completely fair comparison since we also were able to offload persistent storage to Google Cloud, which is one of the harder problems IMO. I think Mesos has improved since then, but it always felt like they were a bit behind. In general Kubernetes feels like it is designed by people with relevant experience. Especially compared to our earlier experiments with Docker Compose files. People are praising their simplicity, but they left us solving a lot of hard problems that Kubernetes solves for us better than we could have done.
- manigandham 9y ago
  Why is it sad? It's great that we can finally standardize and use a single powerful system that is very capable but also improving quickly.
smoyer 9y ago
When we started looking for a container orchestration system, we naturally used Google Trends (https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes https://trends.google.com/trends/explore?date=today%205-y&q=...). Kubernetes reached their 1.0 release shortly before we were ready to start using the system and a lot of the features added since then eliminated so of our other problems (e.g. configmaps/secrets combined with the existing service discovery pretty much eliminated our plans to use Consul). I've been taking a second look at Mesos recently and found that I didn't really grok it the first time I looked at it. In any case, I think your assessment is correct ("Just migrate to Kubernetes" - https://trends.google.com/trends/explore?date=today%205-y&q=docker%20swarm,kubernetes,mesos https://trends.google.com/trends/explore?date=today%205-y&q=...).

robbyt 9y ago

Docker is a amazing tool, but I think the technical design and overall strategy for Swarm wasn't very well executed. Moving to k8s is a smart thing for them, because it's objectively better for real production use. In our tests about a year ago, swarm started showing serious networking and cluster synchronization problems with cluster sizes over 30 nodes (physical servers), on a fast, reliable LAN. I've heard similar stories from another big Docker customer- Docker support promised them that improving performance of Swarm and fixing scaling issues are the focus of "the next version", but they never came. This company is now moving to k8s.

BretFisher 9y ago

Could it be that the teams are simply focused on adding K8s support and getting the Docker EE out of Beta? Public statement on their blog after the K8s announcement in EU: "But it’s equally important for us to note that Swarm orchestration is not going away. Swarm forms an integral cluster management component of the Docker EE platform; in addition, Swarm will operate side-by-side with Kubernetes in a Docker EE cluster, allowing customers to select, based on their needs, the most suitable orchestration tool at application deployment time." https://blog.docker.com/2017/11/swarm-orchestration-in-docker-enterprise-edition/ https://blog.docker.com/2017/11/swarm-orchestration-in-docke... There's still plenty of PR's and activity in the SwarmKit and Libnetwork repos: https://github.com/docker/swarmkit/pulse/monthly https://github.com/docker/swarmkit/pulse/monthly https://github.com/docker/libnetwork/pulse/monthly https://github.com/docker/libnetwork/pulse/monthly