I was in a similar situation a few months ago. I had previous positive experience with k8s and some negative experience with nomad, so I was torn between k3s/k

I was in a similar situation a few months ago. I had previous positive experience with k8s and some negative experience with nomad, so I was torn between k3s/k0s and just docker stack+swarm. I didn't want to install anything as I started with a minuscole node (5$ digital ocean node). In the end I went with docker stack + machine + swarm + docker flow proxy and it's been pretty smooth sailing. I don't need anything else installed on the machine, I can develop stacks locally with docker compose, deploy them to swarm and I can even add replication on multiple machines for services that need it. I manage secrets with stackoverflow's blackbox and the secrets live in the repository encrypted. I can easily run commands on the local or remote stack via docker machine. I cheated with cron jobs and I just synchronise /etc/cron on all hosts with a directory in my repo. The commands are mostly running commands inside containers. I have one stack for redis, postgres, docker registry and multiple stacks for other applications. The flow to deploying stuff is basically: generate id from git hash, build docker image, push to self hosted registry, stack deploy with the git hash as version. A few caveats I found: - Dependencies with docker health checks will take the duration of the health checks to become available to services needing them. Either you skip health checks or setup things so that infra is running before your services. - In order to use docker-machine on multiple machines / with different contributors you need to export the key used to setup the machine and export it (there is a npm package that works very well, but it would be nice to have it natively). - You have to cleanup the docker registry manually every once and then (I had to write a cron job for that) - I'm unsure about the future of docker, albeit things work pretty well as they are.