6 ms·

Error-correction in audio CDs can correct errors up to a specific threshold of corruption. Additionally, there is no way to know that you've passed that thresh

Error-correction in audio CDs can correct errors up to a specific threshold of corruption. Additionally, there is no way to know that you've passed that threshold -- the drive will just return bits, with no way to guarantee that they're correct. A better description for this is "passive" error-correction. To try to get a 100% bit-perfect rip, it's preferable to start off with a drive that you know minimizes errors in the initial read process.

randombits0 4y ago
Ah, so audio CDs were engineered to provide best effort data correction instead of dead-sure data correction to serve the audio market instead of the data market. That just goes to show the lack of design foresight. They were still thinking in terms of grooves carved in plastic or a waveform on tape.
- at_a_remove 4y ago
  What do you suppose dead-sure data correction would look like? My guess is that you couldn't fit a lot of information on it, what with the redundancy to make it perfect.
  - Tempest1981 4y ago
    3 copies of each CD, stored in different locations, inserted into different players, interconnected via ...
  - gnramires 4y ago
    With coding dead-sure correct isn't usually much different in capacity -- you usually don't need to sacrifice bandwidth completely to get reliability. In fact, a simple checksum with a good number of bits (e.g. CRC32) will pretty much give dead-sure accuracy (non-adversarial, you can hash e.g. SHA256 to get cryptographic certainty). If errors were evenly distributed (independent and identically distributed bit flips), it's generally very easy and well known how to obtain dead sure error correction, and the bandwidth you get is known as the channel capacity (C) of the medium (usually in bits/bit or bits/symbol) -- a "good code" operating at a rate R < C (under or equal to capacity) will give you this property; good codes, depending on the error rates, require moderately large block sizes (usually from 8b to 1-2kb in practice). Real media like CDs usually have 'burst errors' like scratches that give a whole bunch of successive bit errors; however, there's an interleaving process to "spread" the errors across blocks, i.e. spread the information so it will resist a scratch. In an absolute way, indeed it's impossible to guarantee almost no errors (although you can guarantee almost no undetected errors) -- simply because your CD might be ruined in a way (which I guess isn't all too unlikely). By setting your rate R to a reasonable level above your drive error rates, you can get almost no errors (as few as you like) within those bounds (and fail above). Here's a typical error rate curve for a moderately large (255b) code: https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction#/media/File:RS_BER.png https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor...
- SideQuark 4y ago
  There's no such thing as dead sure error correction, and there cannot be.
  - sp332 4y ago
    You could include a hash of the correct data, which would tell you in the (cosmically) vast majority of cases when your data has been corrupted.
    - defrost 4y ago
      Indeed. That is, however, error detection and not error correction. Once an error is detected you can apply 'correction' to the nearest probably correct value .. which only 'works' for errors under a threshold. Hence the assertion in comment above yours.
    - ars 4y ago
      How many hashes will you include? A single one for the whole CD? That tells you if you have an error, but not where? A 32 bytes hash for every 32 bytes? Then you double your storage requirements. There are actually WAY better algorithms for this than a hash (a hash is meant to handle secrets and cryptography - it's the wrong thing for this purpose). ECC codes can tell you if the data is bad - and you can choose how many bytes this covers, and that can also fix bad data, again, you can choose how many errors per byte you want. If you are curious: https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code
      - ajsnigrutin 4y ago
        For ripping, would it matter? It's probably not a feature that was needed back in time when audio CDs were designed, but in retrospective, getting a notification "bit error was detected when ripping, not sure where, but now you know" would help preservers - they could then find another copy of the CD and get a 100% accurate digital copy of the song (within limits of hashing algos of course).
      - jefftk 4y ago
- orev 4y ago
  If they had foresight, they wouldn’t have made CDs at all. They do not want people ripping them and getting “perfect” copies.
- fuckstick 4y ago
  > That just goes to show the lack of design foresight. You have a skewed perspective of growing up with ubiquitous computing and cheap ICs. This was very much not the case in the early 70s when CDs were designed. Trying to simultaneously design for high fidelity audio and a then completely theoretical demand for data storage would have made no sense and would be unjustifiable scope creep. The spiral groove of a CD is objectively better for a low cost linear playback device - even though it is obviously worse for random data access (though modern DSP mitigated this 15 years later). Engineering is about trade offs. There’s no binary best effort vs dead sure - it’s a matter of degree, unlike the mistaken GP post, CDDA and CDROM both use RS error detection and correction, just the former has less of it - a CDDA (Redbook) can store about 15% more audio than would be possible with the equivalent WAVE files on a CDROM - at the expense of some redundancy, but it still has very robust error correction - otherwise every little tiny piece of dust would cause a skip. And for audio playback - trying to fill in a best guess is the right thing to do - rather than just making the thing spit the disk out with a failure.
kevin_thibedeau 4y ago
Audio CDs also have significantly weaker error correction than CD-ROM.
fuckstick 4y ago
> Additionally, there is no way to know that you've passed that threshold This is not quite right. CDs use a forward error correcting code, and of course that means you can detect errors (how can one correct errors if they can’t detect them?). The CD drive most definitely can distinguish between an error free signal and a marginal one - both at the analog level and the digital level. Most CD drives firmware are a bit inconsistent in their ability to correctly provide error correction information but this is not a fundamental issue with the technology and far from “no way to know” - relying on the drives error detection facilities is what the C2 error detection option in EAC does for instance. There are some drives that do well with this. https://wiki.hydrogenaud.io/index.php?title=EAC_Drive_Options https://wiki.hydrogenaud.io/index.php?title=EAC_Drive_Option...

randombits0 4y ago

Ah, so audio CDs were engineered to provide best effort data correction instead of dead-sure data correction to serve the audio market instead of the data market. That just goes to show the lack of design foresight. They were still thinking in terms of grooves carved in plastic or a waveform on tape.

at_a_remove 4y ago
What do you suppose dead-sure data correction would look like? My guess is that you couldn't fit a lot of information on it, what with the redundancy to make it perfect.
- Tempest1981 4y ago
  3 copies of each CD, stored in different locations, inserted into different players, interconnected via ...
- gnramires 4y ago
  With coding dead-sure correct isn't usually much different in capacity -- you usually don't need to sacrifice bandwidth completely to get reliability. In fact, a simple checksum with a good number of bits (e.g. CRC32) will pretty much give dead-sure accuracy (non-adversarial, you can hash e.g. SHA256 to get cryptographic certainty). If errors were evenly distributed (independent and identically distributed bit flips), it's generally very easy and well known how to obtain dead sure error correction, and the bandwidth you get is known as the channel capacity (C) of the medium (usually in bits/bit or bits/symbol) -- a "good code" operating at a rate R < C (under or equal to capacity) will give you this property; good codes, depending on the error rates, require moderately large block sizes (usually from 8b to 1-2kb in practice). Real media like CDs usually have 'burst errors' like scratches that give a whole bunch of successive bit errors; however, there's an interleaving process to "spread" the errors across blocks, i.e. spread the information so it will resist a scratch. In an absolute way, indeed it's impossible to guarantee almost no errors (although you can guarantee almost no undetected errors) -- simply because your CD might be ruined in a way (which I guess isn't all too unlikely). By setting your rate R to a reasonable level above your drive error rates, you can get almost no errors (as few as you like) within those bounds (and fail above). Here's a typical error rate curve for a moderately large (255b) code: https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction#/media/File:RS_BER.png https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor...
SideQuark 4y ago
There's no such thing as dead sure error correction, and there cannot be.
- sp332 4y ago
  You could include a hash of the correct data, which would tell you in the (cosmically) vast majority of cases when your data has been corrupted.
  - defrost 4y ago
    Indeed. That is, however, error detection and not error correction. Once an error is detected you can apply 'correction' to the nearest probably correct value .. which only 'works' for errors under a threshold. Hence the assertion in comment above yours.
  - ars 4y ago
    How many hashes will you include? A single one for the whole CD? That tells you if you have an error, but not where? A 32 bytes hash for every 32 bytes? Then you double your storage requirements. There are actually WAY better algorithms for this than a hash (a hash is meant to handle secrets and cryptography - it's the wrong thing for this purpose). ECC codes can tell you if the data is bad - and you can choose how many bytes this covers, and that can also fix bad data, again, you can choose how many errors per byte you want. If you are curious: https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code
    - ajsnigrutin 4y ago
      For ripping, would it matter? It's probably not a feature that was needed back in time when audio CDs were designed, but in retrospective, getting a notification "bit error was detected when ripping, not sure where, but now you know" would help preservers - they could then find another copy of the CD and get a 100% accurate digital copy of the song (within limits of hashing algos of course).
    - jefftk 4y ago
orev 4y ago
If they had foresight, they wouldn’t have made CDs at all. They do not want people ripping them and getting “perfect” copies.
fuckstick 4y ago
> That just goes to show the lack of design foresight. You have a skewed perspective of growing up with ubiquitous computing and cheap ICs. This was very much not the case in the early 70s when CDs were designed. Trying to simultaneously design for high fidelity audio and a then completely theoretical demand for data storage would have made no sense and would be unjustifiable scope creep. The spiral groove of a CD is objectively better for a low cost linear playback device - even though it is obviously worse for random data access (though modern DSP mitigated this 15 years later). Engineering is about trade offs. There’s no binary best effort vs dead sure - it’s a matter of degree, unlike the mistaken GP post, CDDA and CDROM both use RS error detection and correction, just the former has less of it - a CDDA (Redbook) can store about 15% more audio than would be possible with the equivalent WAVE files on a CDROM - at the expense of some redundancy, but it still has very robust error correction - otherwise every little tiny piece of dust would cause a skip. And for audio playback - trying to fill in a best guess is the right thing to do - rather than just making the thing spit the disk out with a failure.

kevin_thibedeau 4y ago

Audio CDs also have significantly weaker error correction than CD-ROM.

fuckstick 4y ago

> Additionally, there is no way to know that you've passed that threshold This is not quite right. CDs use a forward error correcting code, and of course that means you can detect errors (how can one correct errors if they can’t detect them?). The CD drive most definitely can distinguish between an error free signal and a marginal one - both at the analog level and the digital level. Most CD drives firmware are a bit inconsistent in their ability to correctly provide error correction information but this is not a fundamental issue with the technology and far from “no way to know” - relying on the drives error detection facilities is what the C2 error detection option in EAC does for instance. There are some drives that do well with this. https://wiki.hydrogenaud.io/index.php?title=EAC_Drive_Options https://wiki.hydrogenaud.io/index.php?title=EAC_Drive_Option...