3 ms·

> How many times can you copy digital media before it degrades beyond error correction? I don't know the error rate per copied bit. I am not sure I understand.

> How many times can you copy digital media before it degrades beyond error correction? I don't know the error rate per copied bit. I am not sure I understand. Isn’t the error rate zero? And wouldn’t you be using checksums to verify perfect copies?

allovernow 7y ago
Can't be zero. At the very least if bits can be randomly flipped during storage from wayward radiation, they can be flipped on copy. Error correction only works if the number of flipped bits are below a threshold. But I don't know the order of magnitude of the error rate, I assume it's quite small given that computers work.
- Spooky23 7y ago
  It can be very close. You can get durability of data equal or greater than something like Amazon S3. I’d be more worried about disasters and budget cuts than cosmic radiation.
neurocline 7y ago
Back in the 2007-2008 timeframe when I added the streaming install/patch/play functionality to World of Warcraft, we also made it operate as an automatic repair system - if a disk block was read that had a hash mismatch, it just downloaded the correct block from the server (and wrote the fixed block to disk, of course). I looked at a month's worth of download logs, and estimated that 1 bit per GB per year flipped on hard disk data for our customers (we had already implemented re-reads before this point, because we had the hashes in place before we did streaming download, and so were pretty sure it was bad data on disk). Even at that fairly low rate, you can imagine that this reduced daily mysterious crashes by a nice amount. And keep in mind that hard disks already have error correction, so bits on magnetic media aren't as stable as you want. And, of course, this was not data at rest, there's lots of Wow play time going on, so lots of reads. I've been told that tape is more durable than spinning hard disks; for the sake of all those enterprise backups, I hope so. As a side effect, this made me increase how much redundancy I used for data I really cared about.

allovernow 7y ago

Can't be zero. At the very least if bits can be randomly flipped during storage from wayward radiation, they can be flipped on copy. Error correction only works if the number of flipped bits are below a threshold. But I don't know the order of magnitude of the error rate, I assume it's quite small given that computers work.

Spooky23 7y ago
It can be very close. You can get durability of data equal or greater than something like Amazon S3. I’d be more worried about disasters and budget cuts than cosmic radiation.

neurocline 7y ago

Back in the 2007-2008 timeframe when I added the streaming install/patch/play functionality to World of Warcraft, we also made it operate as an automatic repair system - if a disk block was read that had a hash mismatch, it just downloaded the correct block from the server (and wrote the fixed block to disk, of course). I looked at a month's worth of download logs, and estimated that 1 bit per GB per year flipped on hard disk data for our customers (we had already implemented re-reads before this point, because we had the hashes in place before we did streaming download, and so were pretty sure it was bad data on disk). Even at that fairly low rate, you can imagine that this reduced daily mysterious crashes by a nice amount. And keep in mind that hard disks already have error correction, so bits on magnetic media aren't as stable as you want. And, of course, this was not data at rest, there's lots of Wow play time going on, so lots of reads. I've been told that tape is more durable than spinning hard disks; for the sake of all those enterprise backups, I hope so. As a side effect, this made me increase how much redundancy I used for data I really cared about.