3 ms·
At a minimum you need backup, regular restore tests, and alerts when backups stop or restore tests fail. Personally I automate restore testing with cron. I hav
by ohthehugemanate 2y ago
At a minimum you need backup, regular restore tests, and alerts when backups stop or restore tests fail.
Personally I automate restore testing with cron. I have a script that picks two random files from the filesystem: an old one (which should be in long term storage) and a new one (should be in the most recent backup run, more or less), and tries restoring them both and comparing md5sums to the live file. I like this for two reasons: 1. it's easy to alert when a cronjob fails, and 2. I always have a handy working snippet for restoring from backups when I inevitably forget how to use the tooling.
IMO alerting is the trickiest part of the whole setup. I've never really gotten that down on my own.
- Helmut10001 2y agoI recently set up email alerting through the syslog agent from Telegraf-Influx-Grafana, where Grafana is used for Email alerting and InfluxDB for filtering for the specific syslogs. On another VM, I used postfix to email logs after cronjob (failed or passed), which also works great.
- amjd 2y agoI use ntfy.sh for sending push notifications from scripts and such. It's open source and free (they have paid plans as well now, but I didn't encounter any limitations in the free plan). Not an endorsement, just a happy user.
- pohuing 2y ago+1 for ntfy: it's also trivial to self host