6 ms·

> PLANNED FEATURES: erasure coding Now that is something I'd really like to see in Linux filesystems. AFAIK the GNU/Linux implementations of ZFS do not support

• throwaway7767 11y ago
  > Now that is something I'd really like to see in Linux filesystems. AFAIK the GNU/Linux implementations of ZFS do not support ECC. Only the Oracle version. When people talk about "ECC" in relation to ZFS, they are generally referring to ECC memory, i.e. RAM that detects and corrects single-bit corruption. This does not need any support from software, it is a hardware thing. So if you have ECC memory, it will "work with ZFS" no matter your OS. Erasure coding here refers to redundancy in the block storage, so you can lose a disk and still maintain access to the data. And I can confirm that the linux ZFS implementation, along with all others (as they're all based on the same codebase), support that.
  • stewartbutler 11y ago
    Unless he is talking about scrubbing and self-healing? In which case, the same thing still applies -- all releases do this as far as I know. See: https://pthree.org/2012/12/11/zfs-administration-part-vi-scrub-and-resilver/ https://pthree.org/2012/12/11/zfs-administration-part-vi-scr...
    • throwaway7767 11y ago
      Well, scrubbing and self-healing are really just the user-facing operations enabled by the erasure coding, so it's really the same thing.
      • usefulcat 11y ago
        AFAIK zfs doesn't use erasure coding. It has a checksum for each block. Together with redundant storage of blocks, this allows it to determine which block is corrupt when two copies of the same block have different contents.
        • jamesblonde 11y ago
          Redundant storage of the meta data - not the actual blocks. But at least you know they're corrupted if the become corrupted ;)
          • usefulcat 11y ago
            Nope, all blocks, unless wikipedia is wrong or I am completely misunderstanding the following: "When a block is accessed, regardless of whether it is data or meta-data, its checksum is calculated and compared with the stored checksum value of what it "should" be. If the checksums match, the data are passed up the programming stack to the process that asked for it; if the values do not match, then ZFS can heal the data if the storage pool provides data redundancy (such as with internal mirroring), assuming that the copy of data is undamaged and with matching checksums." https://en.wikipedia.org/wiki/ZFS https://en.wikipedia.org/wiki/ZFS
            • wmf 11y ago
              "ZFS can heal the data IF the storage pool provides data redundancy" You can configure ZFS either with or without redundancy.
              • usefulcat 11y ago
                Yes, which is why I said "Together with redundant storage of blocks" in my original comment.
• acqq 11y ago
  The problem with the erasure coding implementations: patents. An example: https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speeding-up-computation-erasure-codes.shtml https://www.techdirt.com/articles/20141115/07113529155/paten...
  • coalescence 11y ago
    There are a few different profiles offered in the Ceph Erasure Coded Pool support, afaik they are based on research papers. Assume due diligence done to ensure they can be used openly (ianal etc). http://ceph.com/docs/master/rados/operations/erasure-code-profile/ http://ceph.com/docs/master/rados/operations/erasure-code-pr... edit: Indeed, checked jerasure site and that's gone.
• tw04 11y ago
  I'd love to see it to, but I've yet to see an implementation of EC that wasn't far, far too slow to be used for any kind of performant workload.
• e12e 11y ago
  What ZFS implementations are you talking about? I'm only aware of zfsonlinux which AFAIK is CDDL (and out-of-tree -- but works fine) -- is there a port licensed under GPL?
• acd 11y ago
  There are RozoFS it's a french company that has made a high performance erasure coding fs. https://en.wikipedia.org/wiki/RozoFS https://en.wikipedia.org/wiki/RozoFS