6 ms·
I thought much the same thing. ZFS scratches most of these itches. (Sharing common blocks, it metadata is self-verifying, it's able to serialize a minimal set o
by drbawb 8y ago
I thought much the same thing. ZFS scratches most of these itches. (Sharing common blocks, it metadata is self-verifying, it's able to serialize a minimal set of changes between two snapshots, etc.) Just ship the filesystem images as, well, filesystem images. Plus if you want to go from development to production: you can `zfs-send` your image onto your HA cluster. ZFS makes for a durable & reliable storage subsystem that's been production-grade for many years.
This is essentially what Illumos/SmartOS does, and it seems to work out well for them.
- cyphar 8y agoThe problem is when you have systems that don't have ZFS, or cases where you want to operate on an image without mounting it. Also (from memory) ZFS de-duplication is not transmitted over zfs-send which means that you don't really get ZFS's de-dup when transferring (you do get it for snapshots -- but then we're back to layer de-duplication which is busted when it comes to containers). Don't get me wrong, I'm a huge fan of ZFS but it really doesn't solve all the problems I went through.
- waz0wski 8y agoZFS supports both compression and de-duplication on send streams, the behavior on the receiving side depends on the configuration of the pool+dataset where the data is being received. There used to be some differences in features/behavior depending on the ZFS version in use (Solaris vs FreeBSD vs ZFSoL/OpenZFS) but I believe as of 2018 all ZFS implementations have these features for send streams