3 ms·
The email attachment scenario is a great example of efficiencies gained by de-duping. The problem is that the application needs to be depdup-aware and needs to
by Duff 15y ago
The email attachment scenario is a great example of efficiencies gained by de-duping. The problem is that the application needs to be depdup-aware and needs to write data in a dedup friendly way. The other more important problem is that there are two cost drivers in storage -- capacity and performance.
Storage systems are unaware of the context of the data they store -- it's just a stream of bits in blocks. The email attachment may not be written in a way that allows a dedupe app to detect the duplication.
Microsoft "solved" this particular problem (email attachment bloat) back in Exchange 2003, by using Single-Instance Storage for mail objects.
But in Exchange 2010, they abandoned this technology completely! Why? They reduced overall I/O load enough to allow the application to work on slow SATA drives, and engineered the application to be able to scale out dramatically.
So while single-instance storage is more technically elegant, some an operations standpoint, there are more cost savings opportunities in replacing a $900 600GB 15k SAS/FC disk with a $400 2TB 5.4k SATA disk.