3 ms·

> Two copies would not help to determine which bit is the correct one, you would need three. Once you have these, I guess you would be able to detect corruption

by codemac 10y ago

> Two copies would not help to determine which bit is the correct one, you would need three. Once you have these, I guess you would be able to detect corruption. Am I guessing correct that duplication beyond 3 times would just help with read speed and in the case when a drive fails while one is already dead (e.g. when you are rebuilding)? You're confusing two topics. Integrity vs. availability. 3x copies, RAID4 XOR, RAIDZ (a form of RAID6), etc are about increasing the availability of your data in the face of device failure/partitions. The integrity of you data is computed different ways in different systems, but you can safely think of it in your head as a pile of checksums on top of eachother. There is a half decent blog entry[0] on Oracle's website, but you can mentally model it as having checksum on disk for each block, which then is combined with other blocks into some larger object that is also checksummed, and on and on until you have "end-to-end integrity". It is a feature of all serious storage systems. I'd love if someone who was more familiar with ZFS could respond with how ZFS's internals work for you.. I worked at a direct competitor of Sun's that sued eachother over this stuff, so I actively tried to not learn about ZFS. I regret this. [0]: https://blogs.oracle.com/bonwick/entry/zfs_end_to_end_data https://blogs.oracle.com/bonwick/entry/zfs_end_to_end_data

bcantrill 10y ago
It seems that you might have edited your penultimate sentence, but just to make sure that the record is unequivocally clear on this: NetApp and Sun didn't simply "sue each other", NetApp initiated patent litigation against Sun in East Texas, and Sun countersued as a defensive maneuver.[1] As for your ZFS question, Bonwick's blog is certainly a good source -- though if one is looking for a thorough treatment, I might recommend "Reliability Analysis of ZFS" by Asim Kadav and Abhishek Rajimwale.[2] [1] https://news.ycombinator.com/item?id=9129784 https://news.ycombinator.com/item?id=9129784 [2] http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf
- codemac 10y ago
  I did not edit that part IIRC, I try not to edit anything substantially that I post online. As far as NetApp v. Sun, it was very unfortunate. I could rant for days about how crappy the business reasons were for NetApp going after Sun (zomg coraid!). I use your quote about Oracle being a lawn mower from your usenix presentation all the time. Thank you for your reply.

by codemac 10y ago

bcantrill 10y ago

It seems that you might have edited your penultimate sentence, but just to make sure that the record is unequivocally clear on this: NetApp and Sun didn't simply "sue each other", NetApp initiated patent litigation against Sun in East Texas, and Sun countersued as a defensive maneuver.[1] As for your ZFS question, Bonwick's blog is certainly a good source -- though if one is looking for a thorough treatment, I might recommend "Reliability Analysis of ZFS" by Asim Kadav and Abhishek Rajimwale.[2] [1] https://news.ycombinator.com/item?id=9129784 https://news.ycombinator.com/item?id=9129784 [2] http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf

codemac 10y ago
I did not edit that part IIRC, I try not to edit anything substantially that I post online. As far as NetApp v. Sun, it was very unfortunate. I could rant for days about how crappy the business reasons were for NetApp going after Sun (zomg coraid!). I use your quote about Oracle being a lawn mower from your usenix presentation all the time. Thank you for your reply.