3 ms·

Mail is one of the easier things to backup IMO. It's just a directory if you use Maildir. Just tar it up with other important stuff on your filesystem you proba

Mail is one of the easier things to backup IMO. It's just a directory if you use Maildir. Just tar it up with other important stuff on your filesystem you probably also want to save. No need for fancy "streaming SQLite replication."

postmanpat 5y ago
I recall Litestream citing a ~3s streaming replication lag. What replication lag could tar-on-a-cronjob reasonably achieve for a moderate 5GB mail store? Even tar'ing isn't trivial: I've seen casing of people having missed preserving file permissions, making restoration difficult.
- bombcar 5y ago
  At 5G size you could do something with ZFS and tons of snapshots and zfs send.
  - creshal 5y ago
    Depends on how much latency/what big a time window for data loss between runs you can afford. We went from streaming cyrus-imapd replication to minutely send/receive for availability, but that introduces a 30-90 time window where data can be lost. Fine for us (esp. considering the reduction in ops overhead), but not necessarily for everyone. (Backups are done by rsnapshot pulling from a snapshot, so we have another copy outside ZFS just in case.)
  - jraph 5y ago
    I'd go with rsync for the transfer (+ zfs/btrfs snapshots on the receiving side to keep incremental backups)
  - 3np 5y ago
    Two popular alternatives for this: https://github.com/zrepl/zrepl https://github.com/zrepl/zrepl https://github.com/jimsalterjrs/sanoid/ https://github.com/jimsalterjrs/sanoid/
- creshal 5y ago
  Replication and backups are usually different things. If you need replication, yes, backup tools won't cut it latency wise. But proper backup tools like rsnapshot, borg, etc. pp. have no trouble correctly backing up email servers even at 10 or 100 times your mail store size.
- benbjohnson 5y ago
  Litestream author here. It has a 1s replication lag by default. It will snapshot the whole database periodically but otherwise it only incrementally uploads the new WAL pages (which are LZ4 compressed).

postmanpat 5y ago

I recall Litestream citing a ~3s streaming replication lag. What replication lag could tar-on-a-cronjob reasonably achieve for a moderate 5GB mail store? Even tar'ing isn't trivial: I've seen casing of people having missed preserving file permissions, making restoration difficult.

bombcar 5y ago
At 5G size you could do something with ZFS and tons of snapshots and zfs send.
- creshal 5y ago
  Depends on how much latency/what big a time window for data loss between runs you can afford. We went from streaming cyrus-imapd replication to minutely send/receive for availability, but that introduces a 30-90 time window where data can be lost. Fine for us (esp. considering the reduction in ops overhead), but not necessarily for everyone. (Backups are done by rsnapshot pulling from a snapshot, so we have another copy outside ZFS just in case.)
- jraph 5y ago
  I'd go with rsync for the transfer (+ zfs/btrfs snapshots on the receiving side to keep incremental backups)
- 3np 5y ago
  Two popular alternatives for this: https://github.com/zrepl/zrepl https://github.com/zrepl/zrepl https://github.com/jimsalterjrs/sanoid/ https://github.com/jimsalterjrs/sanoid/
creshal 5y ago
Replication and backups are usually different things. If you need replication, yes, backup tools won't cut it latency wise. But proper backup tools like rsnapshot, borg, etc. pp. have no trouble correctly backing up email servers even at 10 or 100 times your mail store size.
benbjohnson 5y ago
Litestream author here. It has a 1s replication lag by default. It will snapshot the whole database periodically but otherwise it only incrementally uploads the new WAL pages (which are LZ4 compressed).