5 ms·

I've long been sad that no one has yet built a state of the art decoder for the recovery of error riddled CDDA discs. A lot of old CDs are not holding up as we

by nullc 7y ago

I've long been sad that no one has yet built a state of the art decoder for the recovery of error riddled CDDA discs. A lot of old CDs are not holding up as well as advertised, resulting in an actual loss of archived information. The RS decoders used in typical CD players doesn't come anywhere near the theoretical performance possible from the format, they're only used as erasure codes. In particular, list decoding with a prior on the audio continuity (using techniques like the Postfish declipper) should do really well because of the the enormous and channel independent interleave used by CDDA... but I've never seen it detecting. 30 years ago that sort of thing would have been computationally infeasible, but it should be possible now.

daveslash 7y ago
>> A lot of old CDs are not holding up as well as advertised Are you referring to pressed or burned CDs? Or Both?
- hnlmorg 7y ago
  Sadly even pressed CDs have been known to rot[0][1]. Albeit it's not I tend to hear about with audio CDs because (presumably because there are usually other ways to acquire copies) however it is something that worries a lot of retro gamers like myself. It means that many of the Mega CD, Saturn, etc games (for example) have a potential life expectency. For me though, scratched discs affect me more than disc rot. It can be a lottery sometimes when buying retro games because you can't always be sure just how well looked after the disc was (even when you have pictures and the item is marked as tested/working). [0] https://en.wikipedia.org/wiki/Disc_rot https://en.wikipedia.org/wiki/Disc_rot [1] https://blog.discogs.com/en/say-no-to-disc-rot-how-to-look-after-cds/ https://blog.discogs.com/en/say-no-to-disc-rot-how-to-look-a...
  - codetrotter 7y ago
    > even when you have pictures and the item is marked as tested/working I bought a PlayStation 2 second hand a while ago to relive some memories and to play some of the games that I hadn’t yet played. One of the games played perfectly well up until about 60% through the story and then there is a bad scratch on the disc at which point the game freezes. I think most sellers of used games will only pop the disc in and see that it starts, if they even bother to test the game at all. That is reasonable of course — we can’t expect anyone to play through a whole game another time over just to see that the whole thing works as it should before they sell it for a couple of dollars. But yeah it’s a risk and it does suck a little bit.
    - Aengeuad 7y ago
      Had a similar problem when buying a used copy of Metal Gear Solid 1 on the PS1, the game and both discs worked fine but if you died on the second disc while in story mode the game would lock up. Added a bit of extra difficulty at any rate!
    - behringer 7y ago
      If you're buying a used game you can be sure it isn't tested and is sold as is. Even if they say it's tested or guaranteed, that only means they're willing to give you a refund if anything is wrong.
    - rasz 7y ago
      CD data is stored on the "paint" side of the disk https://en.wikipedia.org/wiki/Blu-ray#/media/File:Comparison_CD_DVD_HDDVD_BD.svg https://en.wikipedia.org/wiki/Blu-ray#/media/File:Comparison... , so if the scratch is on the shiny side you can buff it out.
  - byuu 7y ago
    I feel we're already reaching a point where game preservation for older systems like Mega CD and PC Engine CD really need to have multiple rips that then weigh all the rips to try and correct for individual errors in each rip. In fact I'm pretty sure groups like Redump are probably doing just that. Fun related article: https://john-millikin.com/%F0%9F%A4%94/why-i-ripped-the-same-cd-300-times https://john-millikin.com/%F0%9F%A4%94/why-i-ripped-the-same...
  - 8bitsrule 7y ago
    I recently ripped (using a blueray drive) a half-dozen CD-R's I burned 18-20 years ago. All had been tucked away in a Logic case; all ripped fine ... except one, which had several visible mechanical dings (3 of 6 tracks lost). Better than I'd hoped. I've had a handfull of newer CD-Rs go unreadable, for invisible reasons.
byuu 7y ago
> The RS decoders used in typical CD players doesn't come anywhere near the theoretical performance possible from the format, they're only used as erasure codes. Reed-Solomon is fascinating, especially the way both RSPC and CIRC have both P and Q parity. Depending on the order and how many passes you do of each, you can recover more and more data. If you can guess where the error is, you can correct up to twice as many errors. And as a last resort for RSPC, you can potentially brute force corrections via the checksum values. Reed-Solomon error correction ends up being more of an art than a science, and it's why different CD-ROM drives have different success rates at reading damaged discs. Explaining Reed-Solomon would be a fun article, but probably too intense and niche a subject. Tons of linear algebra over galois fields, and intimidating if not overly complex algorithms like Berlekamp-Massey, Chien search, and Forney's formula (some of which can be brute forced these days.) RSPC makes it even more complex with two overlapping channels of parity. CIRC (cross-interleave) makes that even more complex still with differing delay slots per byte of error correcting data. See pages 45 and 46: http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-130.pdf http://www.ecma-international.org/publications/files/ECMA-ST...
- tinus_hn 7y ago
  The same error correction is used in qr codes. To correct errors you have to guess which bits are unsure.
  - nullc 7y ago
    You don't have to guess, in that if there are below a threshold number of errors there is a unique decode which will always be right. And a decoder that uses Berlekamp-Massey can find that unique decode with a lot less work than trying every possibility. :)
Erlich_Bachman 7y ago
Small clarification: "state of the art" just means "the best currently available", not "a perfect and effective solution that handles all the problems and has all the possbile functionality". So if no one has ever built a decoder of a certain quality level, that specific level can by definition not be "state of the art".

by nullc 7y ago

daveslash 7y ago

>> A lot of old CDs are not holding up as well as advertised Are you referring to pressed or burned CDs? Or Both?

hnlmorg 7y ago
Sadly even pressed CDs have been known to rot[0][1]. Albeit it's not I tend to hear about with audio CDs because (presumably because there are usually other ways to acquire copies) however it is something that worries a lot of retro gamers like myself. It means that many of the Mega CD, Saturn, etc games (for example) have a potential life expectency. For me though, scratched discs affect me more than disc rot. It can be a lottery sometimes when buying retro games because you can't always be sure just how well looked after the disc was (even when you have pictures and the item is marked as tested/working). [0] https://en.wikipedia.org/wiki/Disc_rot https://en.wikipedia.org/wiki/Disc_rot [1] https://blog.discogs.com/en/say-no-to-disc-rot-how-to-look-after-cds/ https://blog.discogs.com/en/say-no-to-disc-rot-how-to-look-a...
- codetrotter 7y ago
  > even when you have pictures and the item is marked as tested/working I bought a PlayStation 2 second hand a while ago to relive some memories and to play some of the games that I hadn’t yet played. One of the games played perfectly well up until about 60% through the story and then there is a bad scratch on the disc at which point the game freezes. I think most sellers of used games will only pop the disc in and see that it starts, if they even bother to test the game at all. That is reasonable of course — we can’t expect anyone to play through a whole game another time over just to see that the whole thing works as it should before they sell it for a couple of dollars. But yeah it’s a risk and it does suck a little bit.
  - Aengeuad 7y ago
    Had a similar problem when buying a used copy of Metal Gear Solid 1 on the PS1, the game and both discs worked fine but if you died on the second disc while in story mode the game would lock up. Added a bit of extra difficulty at any rate!
  - behringer 7y ago
    If you're buying a used game you can be sure it isn't tested and is sold as is. Even if they say it's tested or guaranteed, that only means they're willing to give you a refund if anything is wrong.
  - rasz 7y ago
    CD data is stored on the "paint" side of the disk https://en.wikipedia.org/wiki/Blu-ray#/media/File:Comparison_CD_DVD_HDDVD_BD.svg https://en.wikipedia.org/wiki/Blu-ray#/media/File:Comparison... , so if the scratch is on the shiny side you can buff it out.
- byuu 7y ago
  I feel we're already reaching a point where game preservation for older systems like Mega CD and PC Engine CD really need to have multiple rips that then weigh all the rips to try and correct for individual errors in each rip. In fact I'm pretty sure groups like Redump are probably doing just that. Fun related article: https://john-millikin.com/%F0%9F%A4%94/why-i-ripped-the-same-cd-300-times https://john-millikin.com/%F0%9F%A4%94/why-i-ripped-the-same...
- 8bitsrule 7y ago
  I recently ripped (using a blueray drive) a half-dozen CD-R's I burned 18-20 years ago. All had been tucked away in a Logic case; all ripped fine ... except one, which had several visible mechanical dings (3 of 6 tracks lost). Better than I'd hoped. I've had a handfull of newer CD-Rs go unreadable, for invisible reasons.

byuu 7y ago

> The RS decoders used in typical CD players doesn't come anywhere near the theoretical performance possible from the format, they're only used as erasure codes. Reed-Solomon is fascinating, especially the way both RSPC and CIRC have both P and Q parity. Depending on the order and how many passes you do of each, you can recover more and more data. If you can guess where the error is, you can correct up to twice as many errors. And as a last resort for RSPC, you can potentially brute force corrections via the checksum values. Reed-Solomon error correction ends up being more of an art than a science, and it's why different CD-ROM drives have different success rates at reading damaged discs. Explaining Reed-Solomon would be a fun article, but probably too intense and niche a subject. Tons of linear algebra over galois fields, and intimidating if not overly complex algorithms like Berlekamp-Massey, Chien search, and Forney's formula (some of which can be brute forced these days.) RSPC makes it even more complex with two overlapping channels of parity. CIRC (cross-interleave) makes that even more complex still with differing delay slots per byte of error correcting data. See pages 45 and 46: http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-130.pdf http://www.ecma-international.org/publications/files/ECMA-ST...

tinus_hn 7y ago
The same error correction is used in qr codes. To correct errors you have to guess which bits are unsure.
- nullc 7y ago
  You don't have to guess, in that if there are below a threshold number of errors there is a unique decode which will always be right. And a decoder that uses Berlekamp-Massey can find that unique decode with a lot less work than trying every possibility. :)

Erlich_Bachman 7y ago

Small clarification: "state of the art" just means "the best currently available", not "a perfect and effective solution that handles all the problems and has all the possbile functionality". So if no one has ever built a decoder of a certain quality level, that specific level can by definition not be "state of the art".