7 ms·

Well, that's disappointing to hear. I've run into multiple MP3s bit-rotting on my MB air over several years. Perhaps soley the fault of HFS+, but trusting the u

by dilap 10y ago

Well, that's disappointing to hear. I've run into multiple MP3s bit-rotting on my MB air over several years. Perhaps soley the fault of HFS+, but trusting the underlying hardware just doesn't seem like a good idea.

itsoggy 10y ago
Yep, I had a similar thing with audio files corrupting on my old MacBook, I was using time machine to backup and guess what.... the copy in the backup was fried too! Switched back to vinyl for my music, my 20 year old Texhnics 1210 will probably out last my current laptop and the next!
- roryisok 10y ago
  I smell a hipster.
- amdavidson 10y ago
  You switched to a playback means that is inherently destructive to the media as a means to protect against data loss?
- DugFin 10y ago
  The physical vinyl won't outlast your digital, it'll just fail more gracefully, slowly losing fidelity every time you scrape it with a needle. You're just replacing a small chance of catastrophic failure with a guaranteed gradual failure.
  - semi-extrinsic 10y ago
    Wikipedia says "Record wear can be reduced to virtual insignificance, however, by the use of a high-quality, correctly adjusted turntable and tonearm, a high-compliance magnetic cartridge with a high-end stylus in good condition, and careful record handling, with non-abrasive removal of dust before playing and other cleaning if necessary." In other words, record wear is negligible if you buy good equipment and take good care of it. Just like the risk of bit rot, catastrophic crashes etc. is negligible if you buy good storage equipment and take good care of your data (good backups etc.) Perhaps the one big advantage of vinyl over digital is that the shelf life of unused vinyl is larger than a human lifetime. A disk stored on a shelf, however, can suffer damage in many ways and is guaranteed to be very hard to connect to your computer in ten-twenty years.
  - itsoggy 10y ago
    With a little calibration of the TT once in a while it will esily outlast me!
Symbiote 10y ago
I add parity archives to add some redundancy to my photographs. I've done it for years, but I think it's useful not to rely on a filesystem to handle this. For example: par2create -r5 -n2 example.par2 *.jpg creates two files, between them giving 5% redundancy. I think that should be more than enough to repair bitrot within a file, but depending how many photos there are, losing a whole file could be more than 5%. par2verify example.par2 will verify, and par2repair will repair corrupted or missing files. https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive
- indolering 10y ago
  It would be neat if there was a parity scheme fast enough to preserve 2% of of all files on disk. It could even be tucked away behind savings from file-level compression.
- mfukar 10y ago
  > it's useful not to rely on a filesystem to handle this In what way?
  - wtallis 10y ago
    Relatively few filesystems offer thorough data checksumming. Hardly any offer erasure coding. RAID at the filesystem layer is a bit more common, but also more inconvenient. Doing erasure coding at the file archive level rather than in the filesystem gives you the freedom to move your archives onto standard everyday filesystems and devices without silently losing the protection.
    - jen20 10y ago
      You don't need _a lot_ of filesystems to offer it - just the one you use - and if this is important to you, just use ZFS.
      - rovr138 10y ago
        What if I use multiple computers and multiple operating systems want to be able to work on this data on them. Par2 would allow me to create the needed data on any computer and test it on any computer. The other alternative is having a server that's up and running all the time, exposed to the internet (or complicate the setup with a VPN), so I can sync. Operations would take a long time (via the internet) or I would anyway need to transfer the data to my computer, work on it, sync it back. During this time, any protections that ZFS offers are null since anything could happen in my computer and I can't test for it locally. ZFS is great. But it's not the answer to everything.
beagle3 10y ago
Damaged files (that can still be read from disk without error) are more likely than not a result of bad memory. It is incredibly unlikely to have bits mutated on disk without triggering a CRC error -- approx 1 in 4 billion errors would go undetected in a CRC-32; and it has to be spread over more than 32-bits-wide. The fact that you had multiple damaged files basically guarantees that it was a faulty memory (or bus, or controller -- but not actual magnetic media corruption) issue. ZFS can't really help you with that as the data was likely damaged in transit rather than on disk; though with its own 256 bit hash, it is likely to detect those faulty system components earlier than later.
- pstadler 10y ago
  To mitigate the likeliness of damage during transit, it's recommened to use ECC memory with ZFS. I had FreeNAS running on a system with a faulty memory module. ZFS correctly discovered damage in random files I just transferred over. Luckily, it wasn't too late to replace the bad module and repeat the transfer. Lessons learned: - Run MemTest86 on hardware before using it as storage - Use an FS that does checksums Unfortunately, the latter doesn't seem to apply to APFS.
  - voidz 10y ago
    Can you always just buy ECC memory (I have DDR3, I guess it's more affordable now since it's a little bit older), or does something else (cpu, motherboard) need to support it as well?
    - pstadler 10y ago
      You need a compatible motherboard and CPU.
      - tw04 10y ago
        Which is why Intel kinda sucks. It would cost them basically nothing to have ECC enabled on all of their hardware, but they insist on using it as a differentiator between server and desktop parts. AMD (at least in the past), includes it on all parts so that it's up to the consumer to choose.

by dilap 10y ago

itsoggy 10y ago

Yep, I had a similar thing with audio files corrupting on my old MacBook, I was using time machine to backup and guess what.... the copy in the backup was fried too! Switched back to vinyl for my music, my 20 year old Texhnics 1210 will probably out last my current laptop and the next!

roryisok 10y ago
I smell a hipster.
amdavidson 10y ago
You switched to a playback means that is inherently destructive to the media as a means to protect against data loss?
DugFin 10y ago
The physical vinyl won't outlast your digital, it'll just fail more gracefully, slowly losing fidelity every time you scrape it with a needle. You're just replacing a small chance of catastrophic failure with a guaranteed gradual failure.
- semi-extrinsic 10y ago
  Wikipedia says "Record wear can be reduced to virtual insignificance, however, by the use of a high-quality, correctly adjusted turntable and tonearm, a high-compliance magnetic cartridge with a high-end stylus in good condition, and careful record handling, with non-abrasive removal of dust before playing and other cleaning if necessary." In other words, record wear is negligible if you buy good equipment and take good care of it. Just like the risk of bit rot, catastrophic crashes etc. is negligible if you buy good storage equipment and take good care of your data (good backups etc.) Perhaps the one big advantage of vinyl over digital is that the shelf life of unused vinyl is larger than a human lifetime. A disk stored on a shelf, however, can suffer damage in many ways and is guaranteed to be very hard to connect to your computer in ten-twenty years.
- itsoggy 10y ago
  With a little calibration of the TT once in a while it will esily outlast me!

Symbiote 10y ago

I add parity archives to add some redundancy to my photographs. I've done it for years, but I think it's useful not to rely on a filesystem to handle this. For example: par2create -r5 -n2 example.par2 *.jpg creates two files, between them giving 5% redundancy. I think that should be more than enough to repair bitrot within a file, but depending how many photos there are, losing a whole file could be more than 5%. par2verify example.par2 will verify, and par2repair will repair corrupted or missing files. https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive

indolering 10y ago
It would be neat if there was a parity scheme fast enough to preserve 2% of of all files on disk. It could even be tucked away behind savings from file-level compression.
mfukar 10y ago
> it's useful not to rely on a filesystem to handle this In what way?
- wtallis 10y ago
  Relatively few filesystems offer thorough data checksumming. Hardly any offer erasure coding. RAID at the filesystem layer is a bit more common, but also more inconvenient. Doing erasure coding at the file archive level rather than in the filesystem gives you the freedom to move your archives onto standard everyday filesystems and devices without silently losing the protection.
  - jen20 10y ago
    You don't need _a lot_ of filesystems to offer it - just the one you use - and if this is important to you, just use ZFS.
    - rovr138 10y ago
      What if I use multiple computers and multiple operating systems want to be able to work on this data on them. Par2 would allow me to create the needed data on any computer and test it on any computer. The other alternative is having a server that's up and running all the time, exposed to the internet (or complicate the setup with a VPN), so I can sync. Operations would take a long time (via the internet) or I would anyway need to transfer the data to my computer, work on it, sync it back. During this time, any protections that ZFS offers are null since anything could happen in my computer and I can't test for it locally. ZFS is great. But it's not the answer to everything.

beagle3 10y ago

Damaged files (that can still be read from disk without error) are more likely than not a result of bad memory. It is incredibly unlikely to have bits mutated on disk without triggering a CRC error -- approx 1 in 4 billion errors would go undetected in a CRC-32; and it has to be spread over more than 32-bits-wide. The fact that you had multiple damaged files basically guarantees that it was a faulty memory (or bus, or controller -- but not actual magnetic media corruption) issue. ZFS can't really help you with that as the data was likely damaged in transit rather than on disk; though with its own 256 bit hash, it is likely to detect those faulty system components earlier than later.

pstadler 10y ago
To mitigate the likeliness of damage during transit, it's recommened to use ECC memory with ZFS. I had FreeNAS running on a system with a faulty memory module. ZFS correctly discovered damage in random files I just transferred over. Luckily, it wasn't too late to replace the bad module and repeat the transfer. Lessons learned: - Run MemTest86 on hardware before using it as storage - Use an FS that does checksums Unfortunately, the latter doesn't seem to apply to APFS.
- voidz 10y ago
  Can you always just buy ECC memory (I have DDR3, I guess it's more affordable now since it's a little bit older), or does something else (cpu, motherboard) need to support it as well?
  - pstadler 10y ago
    You need a compatible motherboard and CPU.
    - tw04 10y ago
      Which is why Intel kinda sucks. It would cost them basically nothing to have ECC enabled on all of their hardware, but they insist on using it as a differentiator between server and desktop parts. AMD (at least in the past), includes it on all parts so that it's up to the consumer to choose.