3 ms·

I don't know if someone knows a better stack for my fleet of self hosted applications, maybe moving to quadlet would simplify stuff ? Right now I have an Ansib

I don't know if someone knows a better stack for my fleet of self hosted applications, maybe moving to quadlet would simplify stuff ? Right now I have an Ansible playbook responsible for updating my services, in a git repo. The playbook stops changed services, backups their configs and volumes, applies the new docker-compose.yml and other files, and restarts them. If any of them fail to start, or aren't reachable after 3 minutes, it rolls back everything *including the volumes* (using buttervolume, docker volumes as btrfs subvolumes to make snapshots free). I am looking into Kubernetes, but I didn't find a single stack/solution that would do all that this system does. For example I found nothing that can auto rollback on failure *including persistent volumes*. I found Argo Rollback but it doesn't seem to have hooks that would allow me to add the functionality.

rollcat 1y ago
YMMV, no warranty, IMHO, etc. Disclaimer: I haven't used k8s in a long while, mostly because I don't really have a good use case for it. You'd need to slightly rethink rollbacks, express them in terms of always rolling forward. K8s supports snapshots directly (you'd need a CSI driver; https://github.com/topolvm/topolvm https://github.com/topolvm/topolvm, https://github.com/openebs/zfs-localpv https://github.com/openebs/zfs-localpv, or similar). Restores happen by creating a new PVC (dataSource from a VolumeSnapshot). So in case rollout to version N+1 fails, instead of a rollback to N you'd roll forward to N+2 (which itself would be a copy of N, but referencing the new PVC/PV). You'd still have to script that sequence of actions somehow - perhaps back to Ansible for that? Considering there might be some YAML parsing and templating involved. Of course this looks (and likely is) much more complicated, so if your use case doesn't justify k8s in the first place, I'd stick to what already works ;)
- LelouBil 1y ago
  Thanks for the recommendation, I'll look into it. I'm interested in moving to Kubernetes to make use of the templating languages available that are better than plain Ansible jinja2, and also offer features like schema checking. Because my services are pretty integrated together and to avoid having hardcoded values in multiple places my Ansible files are a pain to maintain
  - rollcat 1y ago
    The pain of Ansible+Jinja2 is what eventually pushed me to write judo[1]... It works surprisingly well for what it was built for, but of course has limitations (e.g. there's no direct support for a templating language; you just plug env variables into a script). The idea is, do less, and allow other tools to fill the gaps. There's still a lot on my todo list, like env files, controlling parallelism, canaries/batches, etc. I'm currently doing these things using hacky shell scripts, which I don't like, so I'd prefer moving that into the main binary. But I still prefer it as-is over Ansible. [1]: https://github.com/rollcat/judo https://github.com/rollcat/judo

rollcat 1y ago

YMMV, no warranty, IMHO, etc. Disclaimer: I haven't used k8s in a long while, mostly because I don't really have a good use case for it. You'd need to slightly rethink rollbacks, express them in terms of always rolling forward. K8s supports snapshots directly (you'd need a CSI driver; https://github.com/topolvm/topolvm https://github.com/topolvm/topolvm, https://github.com/openebs/zfs-localpv https://github.com/openebs/zfs-localpv, or similar). Restores happen by creating a new PVC (dataSource from a VolumeSnapshot). So in case rollout to version N+1 fails, instead of a rollback to N you'd roll forward to N+2 (which itself would be a copy of N, but referencing the new PVC/PV). You'd still have to script that sequence of actions somehow - perhaps back to Ansible for that? Considering there might be some YAML parsing and templating involved. Of course this looks (and likely is) much more complicated, so if your use case doesn't justify k8s in the first place, I'd stick to what already works ;)

LelouBil 1y ago
Thanks for the recommendation, I'll look into it. I'm interested in moving to Kubernetes to make use of the templating languages available that are better than plain Ansible jinja2, and also offer features like schema checking. Because my services are pretty integrated together and to avoid having hardcoded values in multiple places my Ansible files are a pain to maintain
- rollcat 1y ago
  The pain of Ansible+Jinja2 is what eventually pushed me to write judo[1]... It works surprisingly well for what it was built for, but of course has limitations (e.g. there's no direct support for a templating language; you just plug env variables into a script). The idea is, do less, and allow other tools to fill the gaps. There's still a lot on my todo list, like env files, controlling parallelism, canaries/batches, etc. I'm currently doing these things using hacky shell scripts, which I don't like, so I'd prefer moving that into the main binary. But I still prefer it as-is over Ansible. [1]: https://github.com/rollcat/judo https://github.com/rollcat/judo