6 ms·

Doesn't ZFS replication send a snapshot? So the underlying data changing doesn't have any impact on the snapshot's view of the world.

by gaadd33 11y ago

LeoPanthera 11y ago
Yes, but after the data has changed, I don't want to waste time sending data in the snapshot that is no longer in the live data. I'd rather just carry on from where it is now. My current system does this just fine.
- masklinn 11y ago
  So you add a large amount of data, you snapshot, you delete that data and at that point you send the snapshot but don't want to send the deleted data? Why not just re-snapshot and send the new snapshot instead of the old one? > And I'm also unsure whether it's even possible interrupt a transfer and resume it later, if the underlying data has changed There is no "underlying data has changed" in a snapshot, it's a revision of the filesystem. As for interrupting and resuming later, Delphix apparently has contributed a resumable zfs send/receive to OpenZFS, though I don't know if it's available in yours or rsync.net's distribution: http://blog.delphix.com/matt/2015/03/25/resumable-zfs-sendreceive/ http://blog.delphix.com/matt/2015/03/25/resumable-zfs-sendre... As for the concerns of sending small files first, I don't think ZFS could even use that: I don't believe it's content-addressed, so if you send a set of blocks from snapshot 1, interrupt and send snapshot 2 instead the blocks common to both will still be re-sent, ZFS only knows the block commonality from snapshots and since snapshot 1 hasn't been pushed its contents can't be used for snapshot 2. Unless the resumable replication support uses content addressing but that doesn't seem to be the case.
  - LeoPanthera 11y ago
    > Why not just re-snapshot and send the new snapshot instead of the old one? Because then you have to start form scratch every time. The snapshots are so large that if you wanted to send a complete one, a large amount of stuff will have changed.

Doesn't ZFS replication send a snapshot? So the underlying data changing doesn't have any impact on the snapshot's view of the world.

by gaadd33 11y ago

Doesn't ZFS replication send a snapshot? So the underlying data changing doesn't have any impact on the snapshot's view of the world.

LeoPanthera 11y ago

Yes, but after the data has changed, I don't want to waste time sending data in the snapshot that is no longer in the live data. I'd rather just carry on from where it is now. My current system does this just fine.

masklinn 11y ago
So you add a large amount of data, you snapshot, you delete that data and at that point you send the snapshot but don't want to send the deleted data? Why not just re-snapshot and send the new snapshot instead of the old one? > And I'm also unsure whether it's even possible interrupt a transfer and resume it later, if the underlying data has changed There is no "underlying data has changed" in a snapshot, it's a revision of the filesystem. As for interrupting and resuming later, Delphix apparently has contributed a resumable zfs send/receive to OpenZFS, though I don't know if it's available in yours or rsync.net's distribution: http://blog.delphix.com/matt/2015/03/25/resumable-zfs-sendreceive/ http://blog.delphix.com/matt/2015/03/25/resumable-zfs-sendre... As for the concerns of sending small files first, I don't think ZFS could even use that: I don't believe it's content-addressed, so if you send a set of blocks from snapshot 1, interrupt and send snapshot 2 instead the blocks common to both will still be re-sent, ZFS only knows the block commonality from snapshots and since snapshot 1 hasn't been pushed its contents can't be used for snapshot 2. Unless the resumable replication support uses content addressing but that doesn't seem to be the case.
- LeoPanthera 11y ago
  > Why not just re-snapshot and send the new snapshot instead of the old one? Because then you have to start form scratch every time. The snapshots are so large that if you wanted to send a complete one, a large amount of stuff will have changed.