7 ms·

I would argue the myriad of alternatives you've suggested as alternatives produce significantly more overhead. You linked Swarm CA renewal operations; this is

by MrOwen 5y ago

I would argue the myriad of alternatives you've suggested as alternatives produce significantly more overhead. You linked Swarm CA renewal operations; this is maybe 1/50 of the functionality of cert-manager. Is there a product like cert-manager that I can run on Swarm that's tightly integrated? Why not have ingress intrinsically linked to the underlying reaources like k8s does? Your suggestion means that someone needs to come up with a methodology and automation to update ingress with Swarm. Yeah, it's obviously possible but imo, Traefik ingress reaources are way easier to manage with config spread across far fewer places. Storage.. I have an nfs storage provider in k8s which allows me to dynamically create storage with lifecycle management included... for free... just because I'm using k8s. DR as in something happens to my house or other technical issues which bring down my cluster. I would posit that it's much easier for me to restore a cluster because it's all yaml and imo more descriptive and capable than swarm yaml syntax. I think you get my point. Yes, you can do it all with Swarm but with things like k3s and k0s, it's just as easy to start using k8s. And way more help, docs, and tooling to boot.

danudey 5y ago
Honestly, this is such a common refrain in open-source communities. "I'm going to use this product/project." "You shouldn't use that, use this instead." "But this product/project has all these features I want." "Sure but you can just build and maintain those features yourself if you want. It's open source, after all!" Or alternately: "Well no one really needs those anyway, it's just people being told that they do." I've been hearing this same refrain for over twenty years now. It seems as though some people don't understand that there's actual value in having a system that just works, rather than one that you have to build and maintain yourself just to emulate the system that works.
- saurik 5y ago
  So, I don't use either of these things, but still tend to read the arguments between people who do, and that isn't what it feels like to me (as again, someone who doesn't use either of these things). Specifically, it feels like you are missing that the people who argue against k8s are saying that it is extremely difficult to configure, and so instead of this clean "some people don't understand that it is nice to have a system that just works", it feels like the argument is actually about a tradeoff between what you think is easier to put together: a single system with a million options that seem to all need to be configured or it doesn't work or a handful of systems that are each easy to understand in separate, but where you will need to do all the glue manually.
- KronisLV 5y ago
  I do partially agree with the point that you're making, but perhaps from a slightly different angle - i'd say that the biggest value that Kubernetes provides (or at least tries to, with varying results), is having a common set of abstractions that allow both reusing skills and also treating different vendors' clusters as very similar resources. > It seems as though some people don't understand that there's actual value in having a system that just works, rather than one that you have to build and maintain yourself just to emulate the system that works. This, however, feels like a lie. It might "just work" because of the efforts of hundreds of engineers that are in the employ of the larger cloud vendors, because they're handling all of the problems in the background, however that will also cost you. It might also "just work" in on prem deployments because you simply haven't run into any problems yet... and in my experience it's not a matter of "Whether?" but rather one of "When?" And when these problems inevitably do surface, you have to ask yourself, whether you have the capacity to debug them and mitigate them. In the case of K3s clusters, or another lightweight Kubernetes distro that might be doable. However, if you're dealing with a large distro like RKE or your own custom one, then you might have to debug this complex, fragmented and distributed system to get to the root cause and then figure out how to solve it. There's definitely a reason why so many DevOps specialists with Kubernetes credentials are so well paid right now. There's also a reason for why many of the enterprise deployments of Kubernetes that i've seen are essentially black holes of manhours, much like implementing Apache Kafka when something like RabbitMQ might have sufficed - sometimes because of chasing after hype and doing CV driven development, other times because the people didn't know enough about the other options out there. There's also a reason for why Docker Swarm is still alive and kicking despite how its maintenance and development have been mismanaged, as well as why HashiCorp Nomad is liked by many. In short: The people who will try to build their own complex systems will most likely fail and will build something undocumented, unmaintainable, buggy and unstable. The people who try to adapt the technologies that are better suited for large scale deployments for their own needs, might find that the complexity grinds their velocity to a halt and makes them waste huge amounts of effort in maintaining this solution. The people who want to make someone else deal with these complexities and just want the benefits of the tech, will probably have to pay appropriately for this, which sometimes is a good deal, but other times is a bad one and also ensures vendor lock. The people who reevaluate what they're trying to do and how much they actually need to do, might sometimes settle for a lightweight solution which does what's necessary, not much more and leads to vaguely positive outcomes.
- ljm 5y ago
  'A system that just works' isn't a universal constant, so we have these discussions because 'just works' means different things to different people. You're no more right or wrong here than anyone else is.
1_player 5y ago
Traefik, for example, handles certificates for your services. I've set it up this morning on our Docker Swarm at work, 5 domains are now served by it with automated ACME certificates. It's not like cert-manager does rocket science that is possible only on k8s
KronisLV 5y ago
> You linked Swarm CA renewal operations; this is maybe 1/50 of the functionality of cert-manager. Is there a product like cert-manager that I can run on Swarm that's tightly integrated? > Why not have ingress intrinsically linked to the underlying reaources like k8s does? Your suggestion means that someone needs to come up with a methodology and automation to update ingress with Swarm. Yeah, it's obviously possible but imo, Traefik ingress reaources are way easier to manage with config spread across far fewer places. Traefik can provide a lightweight ingress for Docker Swarm clusters alongside ensuring most of the certificate related functionality that you might want for web apps: https://traefik.io/ https://traefik.io/ In particular, here's an example of using it with Docker Swarm: https://dockerswarm.rocks/traefik/ https://dockerswarm.rocks/traefik/ It integrates well enough that you can set up listening for particular ports, domain names, set up basicauth, use Let's Encrypt for certificates and do many other things with labels inside of your Docker Compose file! > Storage.. I have an nfs storage provider in k8s which allows me to dynamically create storage with lifecycle management included... for free... just because I'm using k8s. Docker also supports volume plugins, some of which were carried out of the base offering not to create needless bloat (much like what K3s does with many other in tree plugins of the heavyweight Kubernetes distros): https://docs.docker.com/engine/extend/plugins_volume/ https://docs.docker.com/engine/extend/plugins_volume/ Here's a pretty good documentation page on how to set them up and use them: https://docs.docker.com/storage/volumes/#share-data-among-machines https://docs.docker.com/storage/volumes/#share-data-among-ma... You'll find examples of SSHFS, NFS, CIFS/Samba there, which should cover most base use cases. The latter two are supported by the default "local" driver (which feels like a naming failure on their part), so it's safe to say that Docker and Docker Swarm support both NFS and CIFS/Samba out of the box. > DR as in something happens to my house or other technical issues which bring down my cluster. I would posit that it's much easier for me to restore a cluster because it's all yaml and imo more descriptive and capable than swarm yaml syntax. I'm afraid i don't follow here. Here's how you set up a cluster: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ https://docs.docker.com/engine/swarm/swarm-tutorial/create-s... There are also commands for managing nodes within the Swarm: https://docs.docker.com/engine/reference/commandline/node/ https://docs.docker.com/engine/reference/commandline/node/ There are also commands for managing the stacks that are deployed: https://docs.docker.com/engine/reference/commandline/stack/ https://docs.docker.com/engine/reference/commandline/stack/ That's literally it. Verbose YAML doesn't imply any additional quality. If you want, feel free to look at the Compose specification too see all of the supported options: https://docs.docker.com/compose/compose-file/compose-file-v3/ https://docs.docker.com/compose/compose-file/compose-file-v3... With Swarm, i've actually found that it's actually very easy to both manage the cluster, resolve any issues (because of very few moving parts), or even just wipe it all and create it anew with all of the deployments in a few commands. Furthermore, there is a very lovely Ansible Swarm module if you'd like: https://docs.ansible.com/ansible/latest/collections/community/docker/docker_swarm_module.html#ansible-collections-community-docker-docker-swarm-module https://docs.ansible.com/ansible/latest/collections/communit... In short, fewer moving parts equal easier administration and fewer Byzantine failures in this case. > I think you get my point. Yes, you can do it all with Swarm but with things like k3s and k0s, it's just as easy to start using k8s. And way more help, docs, and tooling to boot. K3s and other projects like it are a step in the right direction for Kubernetes in smaller deployments! That said, in my experience, the documentation of Docker Swarm is comparatively better to that of many of the Kubernetes distro projects, simply because of how mature and stable Docker and Docker Swarm both are. I don't doubt that the situation will change in a few years, but until then, Swarm is more than enough.

by MrOwen 5y ago

danudey 5y ago

Honestly, this is such a common refrain in open-source communities. "I'm going to use this product/project." "You shouldn't use that, use this instead." "But this product/project has all these features I want." "Sure but you can just build and maintain those features yourself if you want. It's open source, after all!" Or alternately: "Well no one really needs those anyway, it's just people being told that they do." I've been hearing this same refrain for over twenty years now. It seems as though some people don't understand that there's actual value in having a system that just works, rather than one that you have to build and maintain yourself just to emulate the system that works.

saurik 5y ago
So, I don't use either of these things, but still tend to read the arguments between people who do, and that isn't what it feels like to me (as again, someone who doesn't use either of these things). Specifically, it feels like you are missing that the people who argue against k8s are saying that it is extremely difficult to configure, and so instead of this clean "some people don't understand that it is nice to have a system that just works", it feels like the argument is actually about a tradeoff between what you think is easier to put together: a single system with a million options that seem to all need to be configured or it doesn't work or a handful of systems that are each easy to understand in separate, but where you will need to do all the glue manually.
KronisLV 5y ago
I do partially agree with the point that you're making, but perhaps from a slightly different angle - i'd say that the biggest value that Kubernetes provides (or at least tries to, with varying results), is having a common set of abstractions that allow both reusing skills and also treating different vendors' clusters as very similar resources. > It seems as though some people don't understand that there's actual value in having a system that just works, rather than one that you have to build and maintain yourself just to emulate the system that works. This, however, feels like a lie. It might "just work" because of the efforts of hundreds of engineers that are in the employ of the larger cloud vendors, because they're handling all of the problems in the background, however that will also cost you. It might also "just work" in on prem deployments because you simply haven't run into any problems yet... and in my experience it's not a matter of "Whether?" but rather one of "When?" And when these problems inevitably do surface, you have to ask yourself, whether you have the capacity to debug them and mitigate them. In the case of K3s clusters, or another lightweight Kubernetes distro that might be doable. However, if you're dealing with a large distro like RKE or your own custom one, then you might have to debug this complex, fragmented and distributed system to get to the root cause and then figure out how to solve it. There's definitely a reason why so many DevOps specialists with Kubernetes credentials are so well paid right now. There's also a reason for why many of the enterprise deployments of Kubernetes that i've seen are essentially black holes of manhours, much like implementing Apache Kafka when something like RabbitMQ might have sufficed - sometimes because of chasing after hype and doing CV driven development, other times because the people didn't know enough about the other options out there. There's also a reason for why Docker Swarm is still alive and kicking despite how its maintenance and development have been mismanaged, as well as why HashiCorp Nomad is liked by many. In short: The people who will try to build their own complex systems will most likely fail and will build something undocumented, unmaintainable, buggy and unstable. The people who try to adapt the technologies that are better suited for large scale deployments for their own needs, might find that the complexity grinds their velocity to a halt and makes them waste huge amounts of effort in maintaining this solution. The people who want to make someone else deal with these complexities and just want the benefits of the tech, will probably have to pay appropriately for this, which sometimes is a good deal, but other times is a bad one and also ensures vendor lock. The people who reevaluate what they're trying to do and how much they actually need to do, might sometimes settle for a lightweight solution which does what's necessary, not much more and leads to vaguely positive outcomes.
ljm 5y ago
'A system that just works' isn't a universal constant, so we have these discussions because 'just works' means different things to different people. You're no more right or wrong here than anyone else is.

1_player 5y ago

Traefik, for example, handles certificates for your services. I've set it up this morning on our Docker Swarm at work, 5 domains are now served by it with automated ACME certificates. It's not like cert-manager does rocket science that is possible only on k8s

KronisLV 5y ago

> You linked Swarm CA renewal operations; this is maybe 1/50 of the functionality of cert-manager. Is there a product like cert-manager that I can run on Swarm that's tightly integrated? > Why not have ingress intrinsically linked to the underlying reaources like k8s does? Your suggestion means that someone needs to come up with a methodology and automation to update ingress with Swarm. Yeah, it's obviously possible but imo, Traefik ingress reaources are way easier to manage with config spread across far fewer places. Traefik can provide a lightweight ingress for Docker Swarm clusters alongside ensuring most of the certificate related functionality that you might want for web apps: https://traefik.io/ https://traefik.io/ In particular, here's an example of using it with Docker Swarm: https://dockerswarm.rocks/traefik/ https://dockerswarm.rocks/traefik/ It integrates well enough that you can set up listening for particular ports, domain names, set up basicauth, use Let's Encrypt for certificates and do many other things with labels inside of your Docker Compose file! > Storage.. I have an nfs storage provider in k8s which allows me to dynamically create storage with lifecycle management included... for free... just because I'm using k8s. Docker also supports volume plugins, some of which were carried out of the base offering not to create needless bloat (much like what K3s does with many other in tree plugins of the heavyweight Kubernetes distros): https://docs.docker.com/engine/extend/plugins_volume/ https://docs.docker.com/engine/extend/plugins_volume/ Here's a pretty good documentation page on how to set them up and use them: https://docs.docker.com/storage/volumes/#share-data-among-machines https://docs.docker.com/storage/volumes/#share-data-among-ma... You'll find examples of SSHFS, NFS, CIFS/Samba there, which should cover most base use cases. The latter two are supported by the default "local" driver (which feels like a naming failure on their part), so it's safe to say that Docker and Docker Swarm support both NFS and CIFS/Samba out of the box. > DR as in something happens to my house or other technical issues which bring down my cluster. I would posit that it's much easier for me to restore a cluster because it's all yaml and imo more descriptive and capable than swarm yaml syntax. I'm afraid i don't follow here. Here's how you set up a cluster: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ https://docs.docker.com/engine/swarm/swarm-tutorial/create-s... There are also commands for managing nodes within the Swarm: https://docs.docker.com/engine/reference/commandline/node/ https://docs.docker.com/engine/reference/commandline/node/ There are also commands for managing the stacks that are deployed: https://docs.docker.com/engine/reference/commandline/stack/ https://docs.docker.com/engine/reference/commandline/stack/ That's literally it. Verbose YAML doesn't imply any additional quality. If you want, feel free to look at the Compose specification too see all of the supported options: https://docs.docker.com/compose/compose-file/compose-file-v3/ https://docs.docker.com/compose/compose-file/compose-file-v3... With Swarm, i've actually found that it's actually very easy to both manage the cluster, resolve any issues (because of very few moving parts), or even just wipe it all and create it anew with all of the deployments in a few commands. Furthermore, there is a very lovely Ansible Swarm module if you'd like: https://docs.ansible.com/ansible/latest/collections/community/docker/docker_swarm_module.html#ansible-collections-community-docker-docker-swarm-module https://docs.ansible.com/ansible/latest/collections/communit... In short, fewer moving parts equal easier administration and fewer Byzantine failures in this case. > I think you get my point. Yes, you can do it all with Swarm but with things like k3s and k0s, it's just as easy to start using k8s. And way more help, docs, and tooling to boot. K3s and other projects like it are a step in the right direction for Kubernetes in smaller deployments! That said, in my experience, the documentation of Docker Swarm is comparatively better to that of many of the Kubernetes distro projects, simply because of how mature and stable Docker and Docker Swarm both are. I don't doubt that the situation will change in a few years, but until then, Swarm is more than enough.