3 ms·

We have always been in a 'post-file-system-compression' world. But people want to do things like fit on an SSD, so compression continues to be useful. Note tho

We have always been in a 'post-file-system-compression' world. But people want to do things like fit on an SSD, so compression continues to be useful. Note though that some controllers do compression to reduce write amplification (e.g. the SandForce controllers used to do this). So, by using filesystem compression, you might be increasing write amplification and shortening the SSD lifetime.

rictic 10y ago
Is that increasing write amplification to a meaningful degree? Compressing already compressed data is generally only slightly less efficient than just compressing the data once, isn't it?
- danieldk 10y ago
  That's an interesting question. As far as I understand, yes. If you buy, say an 256GB SSD backed by roughly the same amount of storage in physical cells. Such an SSD stores 256 GB regardless whether it is compressed or not. Now suppose that we have a compressed filesystem and an uncompressed filesystem and that both the filesystem and controller compression rates are 2:1. Now let's compare the two scenarios: 1. With filesystem compression: you could store twice the amount of data (~512GB). However, since the controller is not able to compress the data, all cells are completely full. 2. Without filesystem compression: you can store 256GB data. However, since the controller can compress the data, only half of the actual cells are used. Suppose now that in both setups, the SSD is nearly full and we rewrite or perhaps extend some data. In the former case, the controller has to scavenge for partially-used cells to combine and erase. In the latter case, the SSD controller still has 128GB of pristine cells to write the changed data immediately. Of course, compression was always somewhat of a cheapskate solution (and one of the reasons the SandForce controllers are not liked much). Higher-end SSDs would just have more storage space than logically addressable to have some leeway when the SSD is almost full.
Const-me 10y ago
You have 2MB data. You compress it to 1MB, SandForce controller fails to do anything because already compressed, writes it to flash, and you wear out 1MB of physical flash memory. You don’t compress it, SandForce controller compresses it to 1MB, writes it to flash, and you wear out same 1MB of physical flash memory. How (1) might be increasing write amplification and shortening the SSD lifetime compared to (2)? Aren’t they nearly equivalent?

rictic 10y ago

Is that increasing write amplification to a meaningful degree? Compressing already compressed data is generally only slightly less efficient than just compressing the data once, isn't it?

danieldk 10y ago
That's an interesting question. As far as I understand, yes. If you buy, say an 256GB SSD backed by roughly the same amount of storage in physical cells. Such an SSD stores 256 GB regardless whether it is compressed or not. Now suppose that we have a compressed filesystem and an uncompressed filesystem and that both the filesystem and controller compression rates are 2:1. Now let's compare the two scenarios: 1. With filesystem compression: you could store twice the amount of data (~512GB). However, since the controller is not able to compress the data, all cells are completely full. 2. Without filesystem compression: you can store 256GB data. However, since the controller can compress the data, only half of the actual cells are used. Suppose now that in both setups, the SSD is nearly full and we rewrite or perhaps extend some data. In the former case, the controller has to scavenge for partially-used cells to combine and erase. In the latter case, the SSD controller still has 128GB of pristine cells to write the changed data immediately. Of course, compression was always somewhat of a cheapskate solution (and one of the reasons the SandForce controllers are not liked much). Higher-end SSDs would just have more storage space than logically addressable to have some leeway when the SSD is almost full.

Const-me 10y ago

You have 2MB data. You compress it to 1MB, SandForce controller fails to do anything because already compressed, writes it to flash, and you wear out 1MB of physical flash memory. You don’t compress it, SandForce controller compresses it to 1MB, writes it to flash, and you wear out same 1MB of physical flash memory. How (1) might be increasing write amplification and shortening the SSD lifetime compared to (2)? Aren’t they nearly equivalent?