4 ms·

At least one flash controller, the SandForce SF-1200, was by default doing block-level deduplication of data written to it. ... Based on discussions with Kirk

by sc68cal 15y ago

At least one flash controller, the SandForce SF-1200, was by default doing block-level deduplication of data written to it. ... Based on discussions with Kirk McKusick and the ZFS team, the following is a detailed explanation of why this is a problem for ZFS. For critical metadata (and optionally for user data) ZFS stores up to three copies of each block. rut-roh.

jfoutz 15y ago
Is that bad? It seems like one or two dedupes more than cover the cost of redundant metadata. One great example is a document sent as an email attachement to employees. that crap takes up tons of spool space. Just one win like that more than makes up for triple storage of metadata.
- sc68cal 15y ago
  (Why was I voted down!?!??!) No - what they're talking about is that the controller LIES to ZFS about what blocks are being stored. That throws a huge wrench in ZFS' COW and safety features. If ZFS writes two extra copies of a block, and the SSD deduplicates those two extra without bothering to tell ZFS, then you've got a serious problem. Imagine if the SSD deduplicated all the ditto blocks? (Ditto Blocks) http://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_tape http://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_...
  - jrockway 15y ago
    So? All disks lie to the controller. Have you ever had a spinning disk with a bad sector? When the disk controller notices this, it remaps the sector to some spare space elsewhere on the drive. It doesn't ask the OS if this is OK, it Just Fucking Does It. Abstraction means that things lie to you. It also means that spinning disks can last longer than a month.
    - Symmetry 15y ago
      And HDDs have had RAM catches for a while now, further complicating things. The SATA interface defines an abstraction over the storage system that the hard drive has to live up to. If the OS makes assumptions that aren't part of that contract that the drive doesn't live up to, thats the OS's fault. Separation of concerns is a vital engineering practice, and if some software violates abstraction barriers and things break that's the software's own fault.
  - tbrownaw 15y ago
    How exactly does that count as the disk lying? ZFS assumes that different block addresses with the same data point to different pieces of backing storage and therefore have certain behavior in case of errors, but does the spec actually state that this is a valid assumption? Or is this just a case of ZFS relying on particular behavior from things that are implementation-defined or undefined?
- Duff 15y ago
  The email attachment scenario is a great example of efficiencies gained by de-duping. The problem is that the application needs to be depdup-aware and needs to write data in a dedup friendly way. The other more important problem is that there are two cost drivers in storage -- capacity and performance. Storage systems are unaware of the context of the data they store -- it's just a stream of bits in blocks. The email attachment may not be written in a way that allows a dedupe app to detect the duplication. Microsoft "solved" this particular problem (email attachment bloat) back in Exchange 2003, by using Single-Instance Storage for mail objects. But in Exchange 2010, they abandoned this technology completely! Why? They reduced overall I/O load enough to allow the application to work on slow SATA drives, and engineered the application to be able to scale out dramatically. So while single-instance storage is more technically elegant, some an operations standpoint, there are more cost savings opportunities in replacing a $900 600GB 15k SAS/FC disk with a $400 2TB 5.4k SATA disk.

by sc68cal 15y ago

jfoutz 15y ago

Is that bad? It seems like one or two dedupes more than cover the cost of redundant metadata. One great example is a document sent as an email attachement to employees. that crap takes up tons of spool space. Just one win like that more than makes up for triple storage of metadata.

sc68cal 15y ago
(Why was I voted down!?!??!) No - what they're talking about is that the controller LIES to ZFS about what blocks are being stored. That throws a huge wrench in ZFS' COW and safety features. If ZFS writes two extra copies of a block, and the SSD deduplicates those two extra without bothering to tell ZFS, then you've got a serious problem. Imagine if the SSD deduplicated all the ditto blocks? (Ditto Blocks) http://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_tape http://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_...
- jrockway 15y ago
  So? All disks lie to the controller. Have you ever had a spinning disk with a bad sector? When the disk controller notices this, it remaps the sector to some spare space elsewhere on the drive. It doesn't ask the OS if this is OK, it Just Fucking Does It. Abstraction means that things lie to you. It also means that spinning disks can last longer than a month.
  - Symmetry 15y ago
    And HDDs have had RAM catches for a while now, further complicating things. The SATA interface defines an abstraction over the storage system that the hard drive has to live up to. If the OS makes assumptions that aren't part of that contract that the drive doesn't live up to, thats the OS's fault. Separation of concerns is a vital engineering practice, and if some software violates abstraction barriers and things break that's the software's own fault.
- tbrownaw 15y ago
  How exactly does that count as the disk lying? ZFS assumes that different block addresses with the same data point to different pieces of backing storage and therefore have certain behavior in case of errors, but does the spec actually state that this is a valid assumption? Or is this just a case of ZFS relying on particular behavior from things that are implementation-defined or undefined?
Duff 15y ago
The email attachment scenario is a great example of efficiencies gained by de-duping. The problem is that the application needs to be depdup-aware and needs to write data in a dedup friendly way. The other more important problem is that there are two cost drivers in storage -- capacity and performance. Storage systems are unaware of the context of the data they store -- it's just a stream of bits in blocks. The email attachment may not be written in a way that allows a dedupe app to detect the duplication. Microsoft "solved" this particular problem (email attachment bloat) back in Exchange 2003, by using Single-Instance Storage for mail objects. But in Exchange 2010, they abandoned this technology completely! Why? They reduced overall I/O load enough to allow the application to work on slow SATA drives, and engineered the application to be able to scale out dramatically. So while single-instance storage is more technically elegant, some an operations standpoint, there are more cost savings opportunities in replacing a $900 600GB 15k SAS/FC disk with a $400 2TB 5.4k SATA disk.