4 ms·

Another backup possibility I currently use: ZFS on a backup server (not necessarily ZFS on the system that should be backed up), pull data with rsync on the bac

by tobias2014 13y ago

Another backup possibility I currently use: ZFS on a backup server (not necessarily ZFS on the system that should be backed up), pull data with rsync on the backup host to a ZFS, after that make a snapshot for an "incremental backup". So simplified it's like: rsync -avx remote:/etc /backup/ && zfs snapshot backup@`date` With zfSnap (https://github.com/graudeejs/zfSnap https://github.com/graudeejs/zfSnap) you can tell how long incremental backups/snapshots are kept, "rsync && zfSnap -d -a 1w backup" You can take advantage of the /backup/.zfs/snapshot directory to access all snapshots, built-in compression and possible data deduplication. If you also have ZFS on the remote host, you can use zfs send and zfs receive to transfer the snapshot directly to the backup server, instead of using rsync for the diff.

tekacs 13y ago
rsync.net is an example of a host that does something like this (with daily snapshots) $> ssh rsyncnet ls .zfs/snapshot daily_2014-02-09 daily_2014-02-10 daily_2014-02-11 daily_2014-02-12 daily_2014-02-13 daily_2014-02-14 daily_2014-02-15 They allow you to customise the length of time for which these snapshots are kept, too (IIRC) (at the cost to you of the incremental extra storage)
- res0nat0r 13y ago
  rsync.net looks awesome, but it is just way too expensive for me.
  - cbhl 13y ago
    Have you asked for the HN discount? https://news.ycombinator.com/item?id=6554313 https://news.ycombinator.com/item?id=6554313 (I happen to get a discount for being a prgmr.com user, but IIRC the terms are similar.)
  - rsync 13y ago
    You should email us and ask about the "new customer HN reader discount". Also, note that 1TB accounts are 15c per GB, per month, and 10TB accounts are 7c per GB, per month - all with no traffic or usage costs. This compares very favorably with S3 and blows the mozy pro pricing out of the water[1]. [1] https://mozy.com/product/mozy/business https://mozy.com/product/mozy/business
- rsync 13y ago
  Just a clarification ... All accounts get 7 days (as the parent shows) and all 1TB+ accounts get 7 days + 4 weeks. However, if you want a custom schedule (say, 30 days, 8 weeks, and 6 months) it only costs more if the space they use exceeds your existing account. So, if you have a 100 GB account and you use 60 GB and your fancy snapshot schedule (of which the first 7 days is always free) only uses 35 GB ... then there is no additional cost at all.
jedbrown 13y ago
I do the same with btrfs. I actually tried bup first, but had a series of problems (see mailing list) and switched to btrfs snapshots. My main disk is also btrfs so I send incremental snapshots ("btrfs send -p"; faster than rsync and I can keep using the machine without making the backed-up state inconsistent), but the rsync method is fine for other source file systems.
developuh 13y ago
Where can I learn more about using rsync and ZFS ?

tekacs 13y ago

rsync.net is an example of a host that does something like this (with daily snapshots) $> ssh rsyncnet ls .zfs/snapshot daily_2014-02-09 daily_2014-02-10 daily_2014-02-11 daily_2014-02-12 daily_2014-02-13 daily_2014-02-14 daily_2014-02-15 They allow you to customise the length of time for which these snapshots are kept, too (IIRC) (at the cost to you of the incremental extra storage)

res0nat0r 13y ago
rsync.net looks awesome, but it is just way too expensive for me.
- cbhl 13y ago
  Have you asked for the HN discount? https://news.ycombinator.com/item?id=6554313 https://news.ycombinator.com/item?id=6554313 (I happen to get a discount for being a prgmr.com user, but IIRC the terms are similar.)
- rsync 13y ago
  You should email us and ask about the "new customer HN reader discount". Also, note that 1TB accounts are 15c per GB, per month, and 10TB accounts are 7c per GB, per month - all with no traffic or usage costs. This compares very favorably with S3 and blows the mozy pro pricing out of the water[1]. [1] https://mozy.com/product/mozy/business https://mozy.com/product/mozy/business
rsync 13y ago
Just a clarification ... All accounts get 7 days (as the parent shows) and all 1TB+ accounts get 7 days + 4 weeks. However, if you want a custom schedule (say, 30 days, 8 weeks, and 6 months) it only costs more if the space they use exceeds your existing account. So, if you have a 100 GB account and you use 60 GB and your fancy snapshot schedule (of which the first 7 days is always free) only uses 35 GB ... then there is no additional cost at all.

jedbrown 13y ago

I do the same with btrfs. I actually tried bup first, but had a series of problems (see mailing list) and switched to btrfs snapshots. My main disk is also btrfs so I send incremental snapshots ("btrfs send -p"; faster than rsync and I can keep using the machine without making the backed-up state inconsistent), but the rsync method is fine for other source file systems.

developuh 13y ago

Where can I learn more about using rsync and ZFS ?