8 ms·

Error correction and concealment in the Compact Disc system (1982) [pdf]

73 points by mmastrac 3y ago

tverbeure 3y ago
It’s really amazing how many fallback layers there are in a CD to work around errors. There’s a first Reed-Solomon layer that can theoretically correct 2 errors per 28 byte frame, but they actually only fix 1 error and use the remaing error detection as an erasure code (instead of an error correction code) to make a second layer of Reed-Solomon correction much better due to interleaving. If that fails, they have the audio samples swizzled in a way so that you can do interpolation between neighboring samples, and if that fails, there was yet another mechanism (that I forgot the details about) for a final effort. I reminds me a bit of spacecraft, where there’s so much redundancy and flexibility to make things still work in case of a catastrophic failure. The Galileo spacecraft comes to mind, where they reprogrammed the whole communication mechanism (adding compression etc) after the main antenna failed. (https://en.m.wikipedia.org/wiki/Galileo_project#High_gain_antenna_problem https://en.m.wikipedia.org/wiki/Galileo_project#High_gain_an...)
- mips_r4300i 3y ago
  The Swiss cheese safety model, but in disc form! Hah
password4321 3y ago
Related: yesterday's sub-discussion triggered RE: > Yamaha's old CD-Recorder, CRW-F1 really improved sound quality by abusing Red Book standard by lengthening the pits of audio CDs https://news.ycombinator.com/item?id=37726359#37729766 https://news.ycombinator.com/item?id=37726359#37729766
ben7799 3y ago
I still love CDs. I started buying them around 1990-1991. I started with MP3 in 1996 but kept buying CDs. I think I got an iPod in 2003, I digitized all my CDs by 2004-2005 and I did go through a period from about 2005-2013 where I bought almost all my music as digital files. I never got rid of my CDs though, and in recent years I started listening to them again just like Vinyl LPs. There's still something fun about listening to them, and they always sound great and work really well. I've never had a CD get damaged enough to hear any artifact. It is still a good way to support artists you like, and prices have inverted to the point CDs are affordable and Vinyl LPs are quite ridiculous a lot of the time.
- lttlrck 3y ago
  I love my CD collection, also been collecting since 1989/90, a new one arrived in the mail today and I buy used on eBay frequently. It's a musical guide to my life since then. I also went through the MP3/streaming route but it's not the same experience at all. I like the ceremony.
userbinator 3y ago
The picture on the last page makes a great illustration of how dense electronics integration has become over the years --- all of that, and more, are now possible in a single IC: https://www.nxp.com/docs/en/data-sheet/SAF784X.pdf https://www.nxp.com/docs/en/data-sheet/SAF784X.pdf There was an article here with a good graphical description of CDs a few years ago: https://news.ycombinator.com/item?id=21185897 https://news.ycombinator.com/item?id=21185897 Unfortunately that site is gone, and the archive doesn't have a copy.
- snerbles 3y ago
  Maybe this copy? https://web.archive.org/web/20200322151217/https://byuu.net/compact-discs/structure https://web.archive.org/web/20200322151217/https://byuu.net/...
- dfox 3y ago
  Well, it is certainly a prototype. In the early consumer CD players the whole CD decoding datapath was split into chip with the analog part (servo amplifier, which also produces an error signal that contains the read data) and digital chip with its associated DRAM that does the rest of the decoding. It is somehow surprising how empty the single layer PCB in these early CD players is. Obviously, this level of integration was required for the technology to be commercially viable. The CD somewhat coincides with the time of cost-effective VLSI ASICs, which made this possible. On the other hand, the modern CMOS logic runs so fast and can be so densely integrated, that the 15 years old SAF784x apparently implements the “complex analog RF” part of CD player by using DSP techniques. And the process it is made in is apparently mature enough that it can accommodate various analog functions including high-powered ones so you end up with a complete CD/MP3 player on one ASIC (if you can fit what you want to do into 64kB of ARM7TDMI code and can tolerate the sound quality and EMC implications of the integrated audio output DAC)
nickdothutton 3y ago
Thanks to Philips, who produced such a tremendous range of R&D driven products.
FirmwareBurner 3y ago
Ah, back when Philips was at the forefront of cutting edge innovation.
- mitthrowaway2 3y ago
  Now the flag is carried by its spin-off, ASML.
  - FirmwareBurner 3y ago
    And NXP. Though it would have been nice if Philips would have evolved to be the "European iPhone" company instead of selling lights, shavers and food blenders.
    - josephcsible 3y ago
      > if Philips would have evolved to be the "European iPhone" company They've certainly nailed the evil and user-hostile bit of being an iPhone company. From just 2 weeks ago: https://news.ycombinator.com/item?id=37667266 https://news.ycombinator.com/item?id=37667266 https://news.ycombinator.com/item?id=37594377 https://news.ycombinator.com/item?id=37594377
      - snom380 3y ago
        Hue is spun out of Philips and is only using the Philips trademark. https://en.wikipedia.org/wiki/Signify_N.V https://en.wikipedia.org/wiki/Signify_N.V. (Last time I checked, iPhone doesn't _require_ an iCloud account to work, although it still needs to "phone home" to be activated on setup.)
    - extraduder_ire 3y ago
      They also license the brand name to other manufacturers, making all kinds of dollar store tat. The first thing that clued me into this was reading the back of a set of airpod-like wireless headphones.
OfSanguineFire 3y ago
I still feel awe at the compact disc. When I was a teenager in the mid – late 90s, scanning microscopic pits with a freaking laser beam, and storing hundreds of megabytes on a storage medium, still seemed like a big deal, real cutting edge stuff. And yet that format had been invented well over a decade before. I’m also surprised that if the CD was already invented by the early 1980s and widely sold commercially for music, it took until the early 1990s for applications in computing with CD-ROM. Was the bottleneck here CPUs, RAM, and graphics cards, which would not have been able to handle the amount of data that a CD-ROM could store?
- ben7799 3y ago
  I think some older than me may remember differently, but I'm slightly older than you and I'd actually say I think for people who could afford computers there weren't that many years between getting their first CD-ROM equipped computer and their first CD Audio player. It was within a year or two in my family, and my father was in the computer/internet industry by the mid-80s so had access to stuff pretty early. I think we got our first CD audio player in the family in 1990 and got a 486 with a CD-rom drive in late 1992. CD Audio was expensive at first too.
- ajsnigrutin 3y ago
  To me, CDRW was the thing that changed everything... yeah sure, i could burn a CD some time before that too, but every file, every video, song, etc., was measured literally in money for blank CDs, and it was relatively expensive. CDRW? 2MB file? Meh, just burn it, lend it to a friend, get it back, erase, 50meg file, other friend, get some other random stuff back, etc. Then flash storage became cheap and the CDs slowly died (well, there were iomega zips and other stuff in between, but not really that popular).
  - francoisdevlin 3y ago
    I had a zip disk for all my work in college. It's completely impractical for anything these days, but it was so much fun. Having 100mb in your bag in the era of floppies made you feel like a king.
quercusa 3y ago
There's also some clever coding at the physical level with the way pits are arranged on the disc: https://en.wikipedia.org/wiki/Eight-to-fourteen_modulation https://en.wikipedia.org/wiki/Eight-to-fourteen_modulation https://spectrum.ieee.org/kees-immink-the-man-who-put-compact-discs-on-track https://spectrum.ieee.org/kees-immink-the-man-who-put-compac...
dylan604 3y ago
I don't remember which specific audio CD ripper software it was, but one of my most favoritist features ever was the "Try really hard" checkbox. This was used if you had a disc which was hard to read from either scratches or dirty or whatever. I never looked into exactly what it did, but I had assumed that a normal read just read, but try really hard would make use of the error correction which the math caused the slower reads. To this day, I've been known to write functions called tryReallyHard() even if I haven't made a UI option explicitly for it, yet.
- mrguyorama 3y ago
  I don't think many computer CD drives have ever exposed the error correction to the computer. It's usually handled entirely internally, as the link shows on the last page. What that program was probably doing was trying a bunch of times to read the same area from a bunch of different starting points to see if it could tease out the data with a little luck. >math caused the slower reads. The error correction was ALWAYS being done, and that math was done on a dedicated circuit.
- TacticalCoder 3y ago
  CD rippers have progressed a lot. Now most audio CDs' hashes are stored in public databases and you can rip a CD in a 100% bit-perfect way and if a single bit is off, your checksum won't match the hash other people got in the database. And CD rippers do check the online DB automatically. Many people do archive their CD collection in a bit perfect way nowadays and you know when your rip is perfect (and it usually is).
  - ben7799 3y ago
    Yes, and it's been this way for at least 10 years at this point. My first wave of ripping CDs in the early 2000s this stuff wasn't around yet. But I redid everything around 2013 and ripped them all as FLAC, and by that point EAC and the databases were very good.
- OfSanguineFire 3y ago
  cdparanoia[0], still the standard Linux command-line application for CD ripping, took its name from the fact that it goes heavy on error-correcting. However, among the filesharing community the Windows-only application Exact Audio Copy[1] always enjoyed a stronger reputation for accuracy, and several major private torrent trackers and DC++ hubs required EAC to have been used for rips, which sadly prevented Linux-only users from contributing to the scene. [0] https://en.wikipedia.org/wiki/Cdparanoia https://en.wikipedia.org/wiki/Cdparanoia [1] https://en.wikipedia.org/wiki/Exact_Audio_Copy https://en.wikipedia.org/wiki/Exact_Audio_Copy

73 points by mmastrac 3y ago

It’s really amazing how many fallback layers there are in a CD to work around errors. There’s a first Reed-Solomon layer that can theoretically correct 2 errors per 28 byte frame, but they actually only fix 1 error and use the remaing error detection as an erasure code (instead of an error correction code) to make a second layer of Reed-Solomon correction much better due to interleaving. If that fails, they have the audio samples swizzled in a way so that you can do interpolation between neighboring samples, and if that fails, there was yet another mechanism (that I forgot the details about) for a final effort. I reminds me a bit of spacecraft, where there’s so much redundancy and flexibility to make things still work in case of a catastrophic failure. The Galileo spacecraft comes to mind, where they reprogrammed the whole communication mechanism (adding compression etc) after the main antenna failed. (https://en.m.wikipedia.org/wiki/Galileo_project#High_gain_antenna_problem https://en.m.wikipedia.org/wiki/Galileo_project#High_gain_an...)

mips_r4300i 3y ago
The Swiss cheese safety model, but in disc form! Hah

password4321 3y ago

Related: yesterday's sub-discussion triggered RE: > Yamaha's old CD-Recorder, CRW-F1 really improved sound quality by abusing Red Book standard by lengthening the pits of audio CDs https://news.ycombinator.com/item?id=37726359#37729766 https://news.ycombinator.com/item?id=37726359#37729766

ben7799 3y ago

I still love CDs. I started buying them around 1990-1991. I started with MP3 in 1996 but kept buying CDs. I think I got an iPod in 2003, I digitized all my CDs by 2004-2005 and I did go through a period from about 2005-2013 where I bought almost all my music as digital files. I never got rid of my CDs though, and in recent years I started listening to them again just like Vinyl LPs. There's still something fun about listening to them, and they always sound great and work really well. I've never had a CD get damaged enough to hear any artifact. It is still a good way to support artists you like, and prices have inverted to the point CDs are affordable and Vinyl LPs are quite ridiculous a lot of the time.

lttlrck 3y ago
I love my CD collection, also been collecting since 1989/90, a new one arrived in the mail today and I buy used on eBay frequently. It's a musical guide to my life since then. I also went through the MP3/streaming route but it's not the same experience at all. I like the ceremony.

userbinator 3y ago

The picture on the last page makes a great illustration of how dense electronics integration has become over the years --- all of that, and more, are now possible in a single IC: https://www.nxp.com/docs/en/data-sheet/SAF784X.pdf https://www.nxp.com/docs/en/data-sheet/SAF784X.pdf There was an article here with a good graphical description of CDs a few years ago: https://news.ycombinator.com/item?id=21185897 https://news.ycombinator.com/item?id=21185897 Unfortunately that site is gone, and the archive doesn't have a copy.

snerbles 3y ago
Maybe this copy? https://web.archive.org/web/20200322151217/https://byuu.net/compact-discs/structure https://web.archive.org/web/20200322151217/https://byuu.net/...
dfox 3y ago
Well, it is certainly a prototype. In the early consumer CD players the whole CD decoding datapath was split into chip with the analog part (servo amplifier, which also produces an error signal that contains the read data) and digital chip with its associated DRAM that does the rest of the decoding. It is somehow surprising how empty the single layer PCB in these early CD players is. Obviously, this level of integration was required for the technology to be commercially viable. The CD somewhat coincides with the time of cost-effective VLSI ASICs, which made this possible. On the other hand, the modern CMOS logic runs so fast and can be so densely integrated, that the 15 years old SAF784x apparently implements the “complex analog RF” part of CD player by using DSP techniques. And the process it is made in is apparently mature enough that it can accommodate various analog functions including high-powered ones so you end up with a complete CD/MP3 player on one ASIC (if you can fit what you want to do into 64kB of ARM7TDMI code and can tolerate the sound quality and EMC implications of the integrated audio output DAC)

nickdothutton 3y ago

Thanks to Philips, who produced such a tremendous range of R&D driven products.

FirmwareBurner 3y ago

Ah, back when Philips was at the forefront of cutting edge innovation.

mitthrowaway2 3y ago
Now the flag is carried by its spin-off, ASML.
- FirmwareBurner 3y ago
  And NXP. Though it would have been nice if Philips would have evolved to be the "European iPhone" company instead of selling lights, shavers and food blenders.
  - josephcsible 3y ago
    > if Philips would have evolved to be the "European iPhone" company They've certainly nailed the evil and user-hostile bit of being an iPhone company. From just 2 weeks ago: https://news.ycombinator.com/item?id=37667266 https://news.ycombinator.com/item?id=37667266 https://news.ycombinator.com/item?id=37594377 https://news.ycombinator.com/item?id=37594377
    - snom380 3y ago
      Hue is spun out of Philips and is only using the Philips trademark. https://en.wikipedia.org/wiki/Signify_N.V https://en.wikipedia.org/wiki/Signify_N.V. (Last time I checked, iPhone doesn't _require_ an iCloud account to work, although it still needs to "phone home" to be activated on setup.)
  - extraduder_ire 3y ago
    They also license the brand name to other manufacturers, making all kinds of dollar store tat. The first thing that clued me into this was reading the back of a set of airpod-like wireless headphones.

OfSanguineFire 3y ago

I still feel awe at the compact disc. When I was a teenager in the mid – late 90s, scanning microscopic pits with a freaking laser beam, and storing hundreds of megabytes on a storage medium, still seemed like a big deal, real cutting edge stuff. And yet that format had been invented well over a decade before. I’m also surprised that if the CD was already invented by the early 1980s and widely sold commercially for music, it took until the early 1990s for applications in computing with CD-ROM. Was the bottleneck here CPUs, RAM, and graphics cards, which would not have been able to handle the amount of data that a CD-ROM could store?

ben7799 3y ago
I think some older than me may remember differently, but I'm slightly older than you and I'd actually say I think for people who could afford computers there weren't that many years between getting their first CD-ROM equipped computer and their first CD Audio player. It was within a year or two in my family, and my father was in the computer/internet industry by the mid-80s so had access to stuff pretty early. I think we got our first CD audio player in the family in 1990 and got a 486 with a CD-rom drive in late 1992. CD Audio was expensive at first too.
ajsnigrutin 3y ago
To me, CDRW was the thing that changed everything... yeah sure, i could burn a CD some time before that too, but every file, every video, song, etc., was measured literally in money for blank CDs, and it was relatively expensive. CDRW? 2MB file? Meh, just burn it, lend it to a friend, get it back, erase, 50meg file, other friend, get some other random stuff back, etc. Then flash storage became cheap and the CDs slowly died (well, there were iomega zips and other stuff in between, but not really that popular).
- francoisdevlin 3y ago
  I had a zip disk for all my work in college. It's completely impractical for anything these days, but it was so much fun. Having 100mb in your bag in the era of floppies made you feel like a king.

quercusa 3y ago

There's also some clever coding at the physical level with the way pits are arranged on the disc: https://en.wikipedia.org/wiki/Eight-to-fourteen_modulation https://en.wikipedia.org/wiki/Eight-to-fourteen_modulation https://spectrum.ieee.org/kees-immink-the-man-who-put-compact-discs-on-track https://spectrum.ieee.org/kees-immink-the-man-who-put-compac...

dylan604 3y ago

I don't remember which specific audio CD ripper software it was, but one of my most favoritist features ever was the "Try really hard" checkbox. This was used if you had a disc which was hard to read from either scratches or dirty or whatever. I never looked into exactly what it did, but I had assumed that a normal read just read, but try really hard would make use of the error correction which the math caused the slower reads. To this day, I've been known to write functions called tryReallyHard() even if I haven't made a UI option explicitly for it, yet.

mrguyorama 3y ago
I don't think many computer CD drives have ever exposed the error correction to the computer. It's usually handled entirely internally, as the link shows on the last page. What that program was probably doing was trying a bunch of times to read the same area from a bunch of different starting points to see if it could tease out the data with a little luck. >math caused the slower reads. The error correction was ALWAYS being done, and that math was done on a dedicated circuit.
TacticalCoder 3y ago
CD rippers have progressed a lot. Now most audio CDs' hashes are stored in public databases and you can rip a CD in a 100% bit-perfect way and if a single bit is off, your checksum won't match the hash other people got in the database. And CD rippers do check the online DB automatically. Many people do archive their CD collection in a bit perfect way nowadays and you know when your rip is perfect (and it usually is).
- ben7799 3y ago
  Yes, and it's been this way for at least 10 years at this point. My first wave of ripping CDs in the early 2000s this stuff wasn't around yet. But I redid everything around 2013 and ripped them all as FLAC, and by that point EAC and the databases were very good.
OfSanguineFire 3y ago
cdparanoia[0], still the standard Linux command-line application for CD ripping, took its name from the fact that it goes heavy on error-correcting. However, among the filesharing community the Windows-only application Exact Audio Copy[1] always enjoyed a stronger reputation for accuracy, and several major private torrent trackers and DC++ hubs required EAC to have been used for rips, which sadly prevented Linux-only users from contributing to the scene. [0] https://en.wikipedia.org/wiki/Cdparanoia https://en.wikipedia.org/wiki/Cdparanoia [1] https://en.wikipedia.org/wiki/Exact_Audio_Copy https://en.wikipedia.org/wiki/Exact_Audio_Copy