9 ms·

They do not. It would be difficult to plan correctly if your free disk space is… variable. Example: You have an existing 40 gigabyte file It happened to comp

They do not. It would be difficult to plan correctly if your free disk space is… variable. Example: You have an existing 40 gigabyte file It happened to compress well You delete it and your free disk space goes up by 4 gigabytes. You then write a new 40 gigabyte file that doesn’t compress well Replacing an existing file of the same size just ate an extra 36 gigabytes. How would you plan around that? SSDs should store the bytes given and don’t play fancy games.

anamexis 4y ago
Yes they do. https://www.intel.com/content/www/us/en/support/articles/000006354/memory-and-storage.html https://www.intel.com/content/www/us/en/support/articles/000...
- wmf 4y ago
  Note that these are pre-2017 consumer SSDs. I think SSD compression fell out of favor due to the rise of FDE.
  - ChrisLomont 4y ago
    FDE can be done on the SSD after compression, and often is. Adding life to SSDs is a terribly useful feature
ChrisLomont 4y ago
SSDs could still do it for speed and to write less pages, making the drive last longer, and simply report the uncompressed space as used. They already do all sorts of tricks on pages such as moving them logically, having more internally than they report to use as pages wear out, and so on. Given that, it's good sense to compress if at all possible simply to make the drive live longer. And guess what - I just googled, tons of hits, and this has been done for a long time :) So it makes sense, is done, and is important for modern SSD behavior.
- tgsovlerkhgsel 4y ago
  None of this works if the data is encrypted. I'm surprised (and shocked) that letting unencrypted data hit the disk is still common enough to make such optimizations worth it. Even if you just stick the key in the server's TPM without any sealing, an encrypted disk makes it much easier to deal with e.g. drive returns (for warranty or fault analysis) or disposal.
  - ChrisLomont 4y ago
    Plenty of SSDs implement strong encryption, and there's an entire standard API for it. Why waste cpu on tasks your hardware already does? Bitlocker supports on drive hardware encryption, and I'd be surprised if other major file systems didn't. If I recall, it's a FIPS requirement for data at rest now.
    - tgsovlerkhgsel 4y ago
      > Why waste cpu on tasks your hardware already does? Because the hardware cannot be trusted to do it correctly. IIRC Bitlocker stopped relying on it for this reason. https://www.howtogeek.com/fyi/you-cant-trust-bitlocker-to-encrypt-your-ssd-on-windows-10/ https://www.howtogeek.com/fyi/you-cant-trust-bitlocker-to-en... (see the updates)
    - gspr 4y ago
      > Why waste cpu on tasks your hardware already does? Because the storage hardware might be loyal to someone who is not the owner of the data.
  - goodpoint 4y ago
    > I'm surprised (and shocked) that letting unencrypted data hit the disk is still common enough There's very, very little benefit in encrypting data at a filesystem level in a datacenter if you think about it.
    - tgsovlerkhgsel 4y ago
sp332 4y ago
It's not great, but it's also not unheard of. Tape capacities are often quoted at double the actual storage space, with fine print that says "assuming 50% compression". Also, if compression makes IOPs faster or reduces wear on SSDs, people might not complain so much.
natmaka 4y ago
Isn't it solved by adding an abstraction layer between the storage and the filesystem, the way "vdo" ( https://github.com/dm-vdo/vdo https://github.com/dm-vdo/vdo ) does it?
collegeburner 4y ago
how is this any different than e.g. fs level compression with zstd on btrfs? seems enough people find that useful.
yunohn 4y ago
What do you mean? Even without compression, you have half the problem. Obviously, at scale, it’s all just statistics and planning on trends instead of individual files.
sigmoid10 4y ago
Yes they do. Intel writes this for example (not all drives use it, but some definitely do): >Data compression via encoding algorithms enables a solid state drive (SSD) to write less data, which in turn yields higher write bandwidth. With a significant amount of data being compressible, performance benefits can be substantial. https://www.intel.com/content/www/us/en/support/articles/000006354/memory-and-storage.html https://www.intel.com/content/www/us/en/support/articles/000...

anamexis 4y ago

Yes they do. https://www.intel.com/content/www/us/en/support/articles/000006354/memory-and-storage.html https://www.intel.com/content/www/us/en/support/articles/000...

wmf 4y ago
Note that these are pre-2017 consumer SSDs. I think SSD compression fell out of favor due to the rise of FDE.
- ChrisLomont 4y ago
  FDE can be done on the SSD after compression, and often is. Adding life to SSDs is a terribly useful feature

ChrisLomont 4y ago

SSDs could still do it for speed and to write less pages, making the drive last longer, and simply report the uncompressed space as used. They already do all sorts of tricks on pages such as moving them logically, having more internally than they report to use as pages wear out, and so on. Given that, it's good sense to compress if at all possible simply to make the drive live longer. And guess what - I just googled, tons of hits, and this has been done for a long time :) So it makes sense, is done, and is important for modern SSD behavior.

tgsovlerkhgsel 4y ago
None of this works if the data is encrypted. I'm surprised (and shocked) that letting unencrypted data hit the disk is still common enough to make such optimizations worth it. Even if you just stick the key in the server's TPM without any sealing, an encrypted disk makes it much easier to deal with e.g. drive returns (for warranty or fault analysis) or disposal.
- ChrisLomont 4y ago
  Plenty of SSDs implement strong encryption, and there's an entire standard API for it. Why waste cpu on tasks your hardware already does? Bitlocker supports on drive hardware encryption, and I'd be surprised if other major file systems didn't. If I recall, it's a FIPS requirement for data at rest now.
  - tgsovlerkhgsel 4y ago
    > Why waste cpu on tasks your hardware already does? Because the hardware cannot be trusted to do it correctly. IIRC Bitlocker stopped relying on it for this reason. https://www.howtogeek.com/fyi/you-cant-trust-bitlocker-to-encrypt-your-ssd-on-windows-10/ https://www.howtogeek.com/fyi/you-cant-trust-bitlocker-to-en... (see the updates)
  - gspr 4y ago
    > Why waste cpu on tasks your hardware already does? Because the storage hardware might be loyal to someone who is not the owner of the data.
- goodpoint 4y ago
  > I'm surprised (and shocked) that letting unencrypted data hit the disk is still common enough There's very, very little benefit in encrypting data at a filesystem level in a datacenter if you think about it.
  - tgsovlerkhgsel 4y ago

sp332 4y ago

It's not great, but it's also not unheard of. Tape capacities are often quoted at double the actual storage space, with fine print that says "assuming 50% compression". Also, if compression makes IOPs faster or reduces wear on SSDs, people might not complain so much.

natmaka 4y ago

Isn't it solved by adding an abstraction layer between the storage and the filesystem, the way "vdo" ( https://github.com/dm-vdo/vdo https://github.com/dm-vdo/vdo ) does it?

collegeburner 4y ago

how is this any different than e.g. fs level compression with zstd on btrfs? seems enough people find that useful.

yunohn 4y ago

What do you mean? Even without compression, you have half the problem. Obviously, at scale, it’s all just statistics and planning on trends instead of individual files.

sigmoid10 4y ago

Yes they do. Intel writes this for example (not all drives use it, but some definitely do): >Data compression via encoding algorithms enables a solid state drive (SSD) to write less data, which in turn yields higher write bandwidth. With a significant amount of data being compressible, performance benefits can be substantial. https://www.intel.com/content/www/us/en/support/articles/000006354/memory-and-storage.html https://www.intel.com/content/www/us/en/support/articles/000...