Story of one of the projects I am involved in: We came from Ansible managed deployments of vanilla docker with nginx as single node ingress with another load b

Story of one of the projects I am involved in: We came from Ansible managed deployments of vanilla docker with nginx as single node ingress with another load balancer on top of that. Worked fine, but HA for containers that are only allowed to exist once in the stack was one thing that caused us headaches. Then, we had a workshop for Rancher RKE. Looked promising at the start, but operating it became a headache as we didn't have enough people in the project team to maintain it. Certificates expiring was an issue and the fact that you actually kinda had to baby-sit the cluster was a turn off. We killed the switch to kubernetes and moved back to Ansible + nginX + docker. In the meantime we were toying around with Docker Swarm for smaller scale deployments and inhouse infrastructure. We didn't find anything to not like and are currently moving into that direction. How we do things in Swarm: 1. Monitoring using an updated Swarmprom stack (https://github.com/neuroforgede/swarmsible/tree/master/environments/test/test-swarm/stacks/02_monitoring https://github.com/neuroforgede/swarmsible/tree/master/envir...) 2. Graphical Insights into the Cluster / Debugging -> Portainer 3. Ingress: Treafik together with tecnativa/docker-socket-proxy so that traefik does not have to run on the managers 4. Container Autoscaling: did not need it yet for our internal installations as well as our customer deployments on bare metal, but we would go for a solution based on prometheus metrics, similar to https://github.com/UnclePhil/ascaler https://github.com/UnclePhil/ascaler 5. Hardware Autoscaling: We would build a custom script for this based on prometheus that automatically orders servers of Hetzner using their hcloud-cli 6. Volumes: Hetzner Cloud Plugin, see https://github.com/costela/docker-volume-hetzner https://github.com/costela/docker-volume-hetzner - Looking forward to CSI support though. 7. Load Balancer + SSL: in front of the Swarm using our Cloud Provider Reasons that we would dabble in k8s again: 1. A lot of projects are k8s only (see OpenFaaS for example) 2. Finer grained control for User permissions 3. Service Mesh to introduce service accounts without requiring to go through a custom proxy