3 ms·
I’m setting up 3-2-1-ish backups for my infra of 3 hosts, and definitely leaning towards Restic + Backrest. All my hosts run the same CoreOS setup (https://git
by ebrahimh 17d ago
I’m setting up 3-2-1-ish backups for my infra of 3 hosts, and definitely leaning towards Restic + Backrest.
All my hosts run the same CoreOS setup (https://github.com/ebrahim37/infra-template https://github.com/ebrahim37/infra-template), where container volumes are placed in one central volumes/ folder and that is the only thing I have to backup.
I plan to implement it like this:
vps1:
- restic container with custom sh entrypoint that will backup volumes/ to homelab every 24 hours
homelab:
- backrest container, to back up volumes/, do prune/check, replicate repo to offsite
- rest-server container, will store backups from vps1, homelab, offsite
offsite:
- restic container, backs up volumes/ to homelab every 24 hours
- rest-server container, store copy of backups from homelab
Only caveat is backing up databases, will either have to do: stop container, backup volume/database-data, start container; or use pg dump etc.
The deduplication is nice, you can have a snapshot for each week of the past year without crazy storage cost
- codys 17d agoIf your system has a way to take consistent snapshots in the filesystem (btrfs, zfs) or volume manager (lvm2, perhaps with a bit of filesystem support to obtain fs consistency), that can be used to avoid database downtime, if desired. https://www.postgresql.org/docs/current/backup-file.html https://www.postgresql.org/docs/current/backup-file.html
- ray_v 16d ago[flagged]
- zenoprax 16d agoI've been trying to find a solution for this too! I was considering using Rclone but too many things are using SQLite for me to trust rsync. I was also going to go with CoreOS but I'm leaning towards Fedora Cloud now in case I need to manage things a bit more (and "auto updating" is not something I want as that suggests auto rebooting). Your secrets.yaml makes me nervous though - too easy to miss a key and leave something exposed. Why not just add the whole file to the vault?
- ebrahimh 16d agoI like CoreOS because of the fact that all config, etc files, sysctls are in one place. Previously, I was using artix and had this “etc” directory[1] checked in to keep track of system configuration, but there was no good way to keep track of config drift (other than remembering to update this dir). Haven’t gone through a CoreOS update yet (been using for ~2 months), but doubtful it would break anything. I’ve tested to make sure all my containers shutdown gracefully etc. The SOPS (secrets.yaml) pattern is more common in NixOS configs, and I found it works nicely here too. In the artix setup, I had a bunch of .example files strewn around [2], which I had to remember to sync with the real versions. Encrypting a key is just prefixing it with “enc_priv_”, SOPS will encrypt and decrypt it automatically. I keep “public” values plaintext to maybe help someone setting this up for themselves. Just have to double-check git diff before committing. [1] https://github.com/ebrahim37/infra-template/tree/00eccff06aed3b65287ab348fb4a725dccd51986/etc https://github.com/ebrahim37/infra-template/tree/00eccff06ae... [2] https://github.com/ebrahim37/infra-template/blob/00eccff06aed3b65287ab348fb4a725dccd51986/rybbit/env/backend.env.example https://github.com/ebrahim37/infra-template/blob/00eccff06ae...
- rsync 16d agoAre you aware of sqlite3-rsync ? It is a tool that was created by the author of sqlite. It does just what you would expected to do.