3 ms·

It also takes a lot of RAM. 380 bytes for each de-dupe entry (block pointer) IIRC? I think it translated to something like 1GB of RAM per TB of storage assuming

by ssmoot 16y ago

It also takes a lot of RAM. 380 bytes for each de-dupe entry (block pointer) IIRC? I think it translated to something like 1GB of RAM per TB of storage assuming half of it was de-duped. It's really a useless feature when you do the math. Unless you're running a small amount of storage with a very VERY large amount of basically identical volumes (think Amazon) you'll spend more on memory than you did on disk and once you get to that point you'd have almost certainly been better off spending that money on SSDs and using any extra memory for primary cache.

Andys 16y ago
Thats my point - its very useful with SSDs, because you don't need all the RAM, because SSD is so fast at random seek.
- ssmoot 16y ago
  I forget the exact names, but go with it. :-) ZFS has two tables: A Block Pointer Table. This is your hashes of blocks and stays in RAM to let you know where on disk you'll find a particular block. It's also referenced during writing to ensure that the COW operations are transactional. The DDT (DataDedupeTable) is similar, but points to de-duped blocks. You need enough RAM to keep this in memory at all times. Otherwise for every write you'll have to scan through the entire table on disk, find matching blocks, then modify it. You'll also have to scan through it for every read to see if the block you're looking for is de-duped. I've probably mixed up the details. I haven't read the source, just used it. But that's my (maybe flawed) understanding of how it works. Bottom line though, you DO need enough RAM to keep the DDT in memory. If you don't you'll see SEVERE thrashing and SSD or no SSD your disk performance will slow down from thousands of IOPS to tens of IOPS or lower. This exact thing happened to us with Sun's 7310, loaded with the Sun specified SSDs. You need to be very sure you have the hardware for de-dupe, and even then the payoff is so small... The only real use-case I can think of is if you're running a VPS.

by ssmoot 16y ago

Andys 16y ago

Thats my point - its very useful with SSDs, because you don't need all the RAM, because SSD is so fast at random seek.

ssmoot 16y ago
I forget the exact names, but go with it. :-) ZFS has two tables: A Block Pointer Table. This is your hashes of blocks and stays in RAM to let you know where on disk you'll find a particular block. It's also referenced during writing to ensure that the COW operations are transactional. The DDT (DataDedupeTable) is similar, but points to de-duped blocks. You need enough RAM to keep this in memory at all times. Otherwise for every write you'll have to scan through the entire table on disk, find matching blocks, then modify it. You'll also have to scan through it for every read to see if the block you're looking for is de-duped. I've probably mixed up the details. I haven't read the source, just used it. But that's my (maybe flawed) understanding of how it works. Bottom line though, you DO need enough RAM to keep the DDT in memory. If you don't you'll see SEVERE thrashing and SSD or no SSD your disk performance will slow down from thousands of IOPS to tens of IOPS or lower. This exact thing happened to us with Sun's 7310, loaded with the Sun specified SSDs. You need to be very sure you have the hardware for de-dupe, and even then the payoff is so small... The only real use-case I can think of is if you're running a VPS.