5 ms·

If you plan to use it for your personal traditional storage HDD, you have to keep in mind that it will render most (if not all) data recovery tools naught. Tool

If you plan to use it for your personal traditional storage HDD, you have to keep in mind that it will render most (if not all) data recovery tools naught. Tools such as PhotoRec rely on the file headers to determine the file type to recover the data. I have a long-term storage HDD that I thought I would never need data recovery on it until I fucked it up good by a wrong partitioning command. Normally those types of mistakes can be recovered by TestDisk but not that time. I realized my only hope was to use PhotoRec to recover what I can from the garbage that I overwrote on the disk. Thanks to PhotoRec I could recover most of them with messed up names. I was so thankful I didn't use any fancy compression algorithm. Tools such as LVM, SSD backing storage, btrfs, compression and stuff are all nice when you understand their limitations. For now, I don't use all of my storage so I just create ext3, ext4, and exFAT partitions to store my data for the maximal chance of recovery, whether it is due to hardware or software.

zokier 9y ago
On the other hand if you are using full-disk encryption then you will be fucked anyways.
- Arbalest 9y ago
  Depends. If you mean soft compression (not ATA level crypto) you may be able to back up the header externally and still decrypt it. The data will be scrambled where it would have been scrambled anyway, and unmodified regions will remain unmodified.
vardump 9y ago
Using something like VDO for personal storage would be pretty pointless. Unless, of course, you have a ton of similar virtual machines or other data with high amount of 4 kB block size aligned redundancy.
- globalc 9y ago
  If your data directories have much redundancy (so deduplication can help), or are nicely compressable, then VDO can also make sense for home setups. Like companies, you would consider things like "does it make sense for my kind of data", "do I have enough CPU cycles/ram to spend on getting storage down to some degree" and "am I ok with the additional layer, so more complicated setup".
- Arbalest 9y ago
  imo VDO makes sense where you deal with high speed, low capacity media. I've used BTRFS compression to hold huge disk images in RAM or an SSD while I fully strip/TRIM and repack them (say from RAW to qcow2 or similar). Of course this is usually a case in which I've imported a physical disk image of a 500GB drive, but the actual data on it was only ~50GB or less. Edit: Sorry not the disk image, the temporary overlay for zero blanking. But sometimes smaller disk images for the whole thing. Recently I've also experimentally been holding some older games in RAM on a compressed image.
ashleyn 9y ago
I'm also old enough to remember how buggy DoubleSpace was and, truth be told, that still kinda puts me off disk compression.
- bhouston 9y ago
  NTSF compression is pretty transparent and robust. I usually have 60% of my main disk compressed (laptop, I need the space especially with multiple Visual Studios-installed) for many years and no failures in any way.
  - jwandborg 9y ago
    As I understand it you've never had to recover your compressed files.
    - bhouston 9y ago
      I run off Google Drive or GitHub for all files. I think most people should. Local data is ephemeral.
amelius 9y ago
On the other hand, you should make backups anyway.
wmf 9y ago
I'm not saying you're wrong, but "what about data recovery" has been used to argue against every advance in storage, even massive ones like SSDs. Have backups and unshackle yourself from fear.

zokier 9y ago

On the other hand if you are using full-disk encryption then you will be fucked anyways.

Arbalest 9y ago
Depends. If you mean soft compression (not ATA level crypto) you may be able to back up the header externally and still decrypt it. The data will be scrambled where it would have been scrambled anyway, and unmodified regions will remain unmodified.

vardump 9y ago

Using something like VDO for personal storage would be pretty pointless. Unless, of course, you have a ton of similar virtual machines or other data with high amount of 4 kB block size aligned redundancy.

globalc 9y ago
If your data directories have much redundancy (so deduplication can help), or are nicely compressable, then VDO can also make sense for home setups. Like companies, you would consider things like "does it make sense for my kind of data", "do I have enough CPU cycles/ram to spend on getting storage down to some degree" and "am I ok with the additional layer, so more complicated setup".
Arbalest 9y ago
imo VDO makes sense where you deal with high speed, low capacity media. I've used BTRFS compression to hold huge disk images in RAM or an SSD while I fully strip/TRIM and repack them (say from RAW to qcow2 or similar). Of course this is usually a case in which I've imported a physical disk image of a 500GB drive, but the actual data on it was only ~50GB or less. Edit: Sorry not the disk image, the temporary overlay for zero blanking. But sometimes smaller disk images for the whole thing. Recently I've also experimentally been holding some older games in RAM on a compressed image.

ashleyn 9y ago

I'm also old enough to remember how buggy DoubleSpace was and, truth be told, that still kinda puts me off disk compression.

bhouston 9y ago
NTSF compression is pretty transparent and robust. I usually have 60% of my main disk compressed (laptop, I need the space especially with multiple Visual Studios-installed) for many years and no failures in any way.
- jwandborg 9y ago
  As I understand it you've never had to recover your compressed files.
  - bhouston 9y ago
    I run off Google Drive or GitHub for all files. I think most people should. Local data is ephemeral.

amelius 9y ago

On the other hand, you should make backups anyway.

wmf 9y ago

I'm not saying you're wrong, but "what about data recovery" has been used to argue against every advance in storage, even massive ones like SSDs. Have backups and unshackle yourself from fear.