5 ms·

btrfs only uses CRC32c which is weakish. ZFS is great but not exactly portable. I started to use Borg now for archiving purposes as well, not just backup. For m

by dom0 10y ago

btrfs only uses CRC32c which is weakish. ZFS is great but not exactly portable. I started to use Borg now for archiving purposes as well, not just backup. For me (low access concurrency, i.e. single or at most "a few" users) that works very well. Portable + strong checksumming + strong crypto + mountable + reasonable speed (with prospect of more) is a good package. It doesn't solve error correction, though.

X86BSD 10y ago
ZFS not portable? Have you ever used it? "ZFS export" on the host system, remove drives, insert drives in new server, "ZFS import" on the new server, It really is that simple.
- dpedu 10y ago
  I think he means portability of the software. What I want to 'ZFS import' on a Windows or macOS machine?
  - justincormack 10y ago
    ZFS on OSX has been revived I believe.
    - drvdevd 10y ago
      I wonder if the recent Linux syscall emulation on Windows would somehow make it possible or easier to port ZFS on Linux to Windows. I know you have the SPL anyway, so maybe with the addition of the Linux POSIX-ish layer in there this could be the case...
      - e12e 10y ago
        Fwiw I've successfully shared my luks-encrypted usb3 zfs-formatted disk to Windows pro on my Surface 4 via a hyper-v vm running Ubuntu and samba. It won't work for all external drives - you need to be able to set the drive as "offline" in device manager under Windows in order to pass it through to the hyper-v vm (and sadly this doesn't appear possible with the sdcard - I had hoped to install Ubuntu on the sdcard and have the option to boot from the sdcard and also boot into the same filsystem under hyper-v).
      - chungy 10y ago
        Short answer: No it wouldn't make it easier to port. Longer answer: The Linux subsystem in Windows 10 only deals with userspace. It doesn't support kernel modules nor changes anything about making Windows drivers. Porting ZFS to Windows is certainly possible, but it will take quite a lot of effort, and the Linux subsystem is irrelevant in that situation.
        drvdevd 10y ago
        Yeah, I figured as much, but was hoping there might be something about the Linux subsystem that would be helpful in porting drivers around, beyond userspace. Obviously, I've yet to actually use it myself.
        gjjrfcbugxbhf 10y ago
        Would a FUSE implementation of ZFS not be possible? (Just wondering I've no idea what's possible here)
        Dylan16807 10y ago
        A FUSE implementation of ZFS exists and works well, and adding FUSE support to the Windows 10 Linux subsystem appears to be reasonably high up on the priority list. That doesn't get you access from Windows programs, but there are some other ways to do FUSE or FUSE-like things on Windows..
        rincebrain 10y ago
        Last I knew, the zfs-fuse codebase hadn't been updated since before feature flags were added to any of the OpenZFS targets, so it's not a particularly well-supported solution...
        problems 10y ago
        It may be fairly straightforward to port the ZFS FUSE to Windows if you use things like Dokan or WinFsp which have the FUSE interface supported fairly well - these would give full access via standard Windows tools.
  - X86BSD 10y ago
    Oh, well it's ported from Solaris to illumos and FreeBSD and to a lesser extent OS X and Linux. So I'm still confused about the portability claim.
    - tux1968 10y ago
      The license incompatibility has kept it out of the mainline kernel for Linux, so it's not really a viable option there in many situations. Linux is definitely lacking in this department.
      - rleigh 10y ago
        That's a minor concern and unrelated to the portability claim. That comes down to choice, and is not a technical consideration. I'm using it with Ubuntu 16.04 LTS and 16.10 where it works out of the box. It's most certainly portable to and from Linux and other systems; I've done it personally, and it works a treat.
        tux1968 10y ago
        Minor for some, major for others. And it is technical too, because there are maintenance ramifications for it not being in the mainline kernel, for example how quickly a security patch can be applied. And for others a constriction on which distribution they can move to etc. It just reduces the number of situations where it can be used, even if you find yourself in one where it can.
rleigh 10y ago
Not portable? In comparison to which filesytems? I can easily export a ZFS pool on Linux, physically transport the discs to a FreeBSD or IllumOS server and import the pool. Or to MacOS X, which is also supported (though I haven't tried it unlike the others where it worked perfectly). That's already far ahead of ext, ufs, xfs, jfs, btrfs etc. The only ones offhand which are possibly more portable are fat, hfs, udf and ntfs, and you're not exactly going to want to use them for any serious purpose on a Unix system. ZFS is the most portable and featureful Unix filesystem around at present IMHO.
ak217 10y ago
crc32c is not weakish, and was chosen for a reason: crc32c has widespread hardware acceleration support that remains faster than any hash, and crc32c can be computed in parallel (unlike a hash, it has no hidden state, so you can sum independently computed block checksums to get the overall blob checksum). Bitrot detection doesn't need a cryptographic hash. You may want a hash for other purposes (like if you somehow trust your metadata volume more than your data volume), but that's a separate and slower use case.
- dom0 10y ago
  CRC32c -> I saw this many times fail to detect corruption on message lengths anywhere between a couple kB and a few MB. btrfs blocks are 16 kB iirc, so in range. The longer hashes of ZFS, Borg and so on mean that if it's corrupted I _definitely_ know. Not so confident with CRC32 from experience.
  - ak217 10y ago
    I'm curious about the setting in which you saw these failures, could you elaborate? Unlike a plain checksum, CRC-32C is hardened against bias, which means its distribution is not far from that of an ideal checksum. This means if your bitrot is random and you're using 16KB blocks, you will need to see on the order of ((2 * * 32) * 16KB)=64TB of corrupted data to get a random failure. Modern hard drives corrupt data at a rate of once every few terabytes. TCP without SSL (because of its highly imperfect checksum) corrupts data at a rate of once every few gigabytes. Assuming an extremely bad scenario of a corrupt packet once every 1 GB, in theory you'd need to read more than a zettabyte of data to get a random CRC-32C failure. I'm not doubting that real world performance could be much worse, but I'd like to understand how.
    - planteen 10y ago
      Posts like this are what keep me coming back to HN.
    - vardump 10y ago
      > This means if your bitrot is random and you're using 16KB blocks, you will need to see on the order of ((2 * * 32) * 16KB)=64TB of corrupted data to get a random failure. No, that's what you need to generate one guaranteed failure, when enumerating different random corruption possibilities. Simply because 32-bit number can at most represent 2^32 different states. In practice, you'd have 50% probability to have a collision for every 32 TB... assuming perfect distribution. By the way, 32 TB takes just 4-15 hours to read from a modern SSD. Terabyte is just not so much data nowadays.
johnramsden 10y ago
ZFS is probably one of your better options if you want portability. It works on most Unix like operating systems, and due to its zfs send/recv capability you can send your data all over the place - and can trust that it will actually end up the same on the other end. If that's not portability I don't know what is. If what you're saying is it's not portable to Windows then sure, but compared to most filesystems it's extremely portable.

by dom0 10y ago

X86BSD 10y ago

ZFS not portable? Have you ever used it? "ZFS export" on the host system, remove drives, insert drives in new server, "ZFS import" on the new server, It really is that simple.

dpedu 10y ago
I think he means portability of the software. What I want to 'ZFS import' on a Windows or macOS machine?
- justincormack 10y ago
  ZFS on OSX has been revived I believe.
  - drvdevd 10y ago
    I wonder if the recent Linux syscall emulation on Windows would somehow make it possible or easier to port ZFS on Linux to Windows. I know you have the SPL anyway, so maybe with the addition of the Linux POSIX-ish layer in there this could be the case...
    - e12e 10y ago
      Fwiw I've successfully shared my luks-encrypted usb3 zfs-formatted disk to Windows pro on my Surface 4 via a hyper-v vm running Ubuntu and samba. It won't work for all external drives - you need to be able to set the drive as "offline" in device manager under Windows in order to pass it through to the hyper-v vm (and sadly this doesn't appear possible with the sdcard - I had hoped to install Ubuntu on the sdcard and have the option to boot from the sdcard and also boot into the same filsystem under hyper-v).
    - chungy 10y ago
      Short answer: No it wouldn't make it easier to port. Longer answer: The Linux subsystem in Windows 10 only deals with userspace. It doesn't support kernel modules nor changes anything about making Windows drivers. Porting ZFS to Windows is certainly possible, but it will take quite a lot of effort, and the Linux subsystem is irrelevant in that situation.
      - drvdevd 10y ago
        Yeah, I figured as much, but was hoping there might be something about the Linux subsystem that would be helpful in porting drivers around, beyond userspace. Obviously, I've yet to actually use it myself.
      - gjjrfcbugxbhf 10y ago
        Would a FUSE implementation of ZFS not be possible? (Just wondering I've no idea what's possible here)
        Dylan16807 10y ago
        A FUSE implementation of ZFS exists and works well, and adding FUSE support to the Windows 10 Linux subsystem appears to be reasonably high up on the priority list. That doesn't get you access from Windows programs, but there are some other ways to do FUSE or FUSE-like things on Windows..
        rincebrain 10y ago
        Last I knew, the zfs-fuse codebase hadn't been updated since before feature flags were added to any of the OpenZFS targets, so it's not a particularly well-supported solution...
        problems 10y ago
        It may be fairly straightforward to port the ZFS FUSE to Windows if you use things like Dokan or WinFsp which have the FUSE interface supported fairly well - these would give full access via standard Windows tools.
- X86BSD 10y ago
  Oh, well it's ported from Solaris to illumos and FreeBSD and to a lesser extent OS X and Linux. So I'm still confused about the portability claim.
  - tux1968 10y ago
    The license incompatibility has kept it out of the mainline kernel for Linux, so it's not really a viable option there in many situations. Linux is definitely lacking in this department.
    - rleigh 10y ago
      That's a minor concern and unrelated to the portability claim. That comes down to choice, and is not a technical consideration. I'm using it with Ubuntu 16.04 LTS and 16.10 where it works out of the box. It's most certainly portable to and from Linux and other systems; I've done it personally, and it works a treat.
      - tux1968 10y ago
        Minor for some, major for others. And it is technical too, because there are maintenance ramifications for it not being in the mainline kernel, for example how quickly a security patch can be applied. And for others a constriction on which distribution they can move to etc. It just reduces the number of situations where it can be used, even if you find yourself in one where it can.

rleigh 10y ago

Not portable? In comparison to which filesytems? I can easily export a ZFS pool on Linux, physically transport the discs to a FreeBSD or IllumOS server and import the pool. Or to MacOS X, which is also supported (though I haven't tried it unlike the others where it worked perfectly). That's already far ahead of ext, ufs, xfs, jfs, btrfs etc. The only ones offhand which are possibly more portable are fat, hfs, udf and ntfs, and you're not exactly going to want to use them for any serious purpose on a Unix system. ZFS is the most portable and featureful Unix filesystem around at present IMHO.

ak217 10y ago

crc32c is not weakish, and was chosen for a reason: crc32c has widespread hardware acceleration support that remains faster than any hash, and crc32c can be computed in parallel (unlike a hash, it has no hidden state, so you can sum independently computed block checksums to get the overall blob checksum). Bitrot detection doesn't need a cryptographic hash. You may want a hash for other purposes (like if you somehow trust your metadata volume more than your data volume), but that's a separate and slower use case.

dom0 10y ago
CRC32c -> I saw this many times fail to detect corruption on message lengths anywhere between a couple kB and a few MB. btrfs blocks are 16 kB iirc, so in range. The longer hashes of ZFS, Borg and so on mean that if it's corrupted I _definitely_ know. Not so confident with CRC32 from experience.
- ak217 10y ago
  I'm curious about the setting in which you saw these failures, could you elaborate? Unlike a plain checksum, CRC-32C is hardened against bias, which means its distribution is not far from that of an ideal checksum. This means if your bitrot is random and you're using 16KB blocks, you will need to see on the order of ((2 * * 32) * 16KB)=64TB of corrupted data to get a random failure. Modern hard drives corrupt data at a rate of once every few terabytes. TCP without SSL (because of its highly imperfect checksum) corrupts data at a rate of once every few gigabytes. Assuming an extremely bad scenario of a corrupt packet once every 1 GB, in theory you'd need to read more than a zettabyte of data to get a random CRC-32C failure. I'm not doubting that real world performance could be much worse, but I'd like to understand how.
  - planteen 10y ago
    Posts like this are what keep me coming back to HN.
  - vardump 10y ago
    > This means if your bitrot is random and you're using 16KB blocks, you will need to see on the order of ((2 * * 32) * 16KB)=64TB of corrupted data to get a random failure. No, that's what you need to generate one guaranteed failure, when enumerating different random corruption possibilities. Simply because 32-bit number can at most represent 2^32 different states. In practice, you'd have 50% probability to have a collision for every 32 TB... assuming perfect distribution. By the way, 32 TB takes just 4-15 hours to read from a modern SSD. Terabyte is just not so much data nowadays.

johnramsden 10y ago

ZFS is probably one of your better options if you want portability. It works on most Unix like operating systems, and due to its zfs send/recv capability you can send your data all over the place - and can trust that it will actually end up the same on the other end. If that's not portability I don't know what is. If what you're saying is it's not portable to Windows then sure, but compared to most filesystems it's extremely portable.