6 ms·

FLAC makes lots of smart decisions. One constraint on lossless audio should, of course, be that it's actually lossless (which the test suite confirms), but als

FLAC makes lots of smart decisions. One constraint on lossless audio should, of course, be that it's actually lossless (which the test suite confirms), but also that it makes it easy for the user to confirm the audio streams are the same as the decoded WAV file (`flac -t`) by storing a checksum of the stream. I have often wondered why other media formats don't do a similar thing, especially since changing a media file's tags (which can change the checksum of a file) or name (which makes external verification from txt file difficult) is quite common. I even wrote a utility[0] that uses ffmpeg to hash all ffmpeg compatible bitstreams, and store their hashes in a xattr (yes, with lots of other options to test and compare, etc.), but all media formats should just be as clever (and care as much) to do this natively, like FLAC. I mean -- why not? [0]: https://github.com/kimono-koans/dano https://github.com/kimono-koans/dano

lake_vincent 4y ago
What are the reasons why not? What are the obstacles/drawbacks to the FLAC approach? I have no domain knowledge here, just curious
- lazide 4y ago
  For the most part, no one cares to do so is why. Hell, 99% of all computers are still running on non-checksummed filesystems, if not 99.99%. For the most part, only a tiny minority of people are even aware of bit rot, let alone have anything they simultaneously care enough about to protect against, and have enough ownership of it to attempt to try.
hnuser123456 4y ago
Are there any good ways to tell if audio has been lossy compressed but then re-encoded in a lossless format, or deceptively high lossy bitrate?
- dmitri_ignat 4y ago
  You can view it with a spectrogram, audio which has been lossily encoded will have a telltale cutoff in the high frequencies. The lossier the encoding, the lower the frequency ceiling will be.
  - sdenton4 4y ago
    Another thing to look for is deleted or very flat energy between harmonics. Reducing bit rate in inter harmonic regions is most of what's meant by 'psychoacoustics.' I imagine low pass filtering depends a bit on which compression you're using. None of these tricks will work for detecting GAN compressed audio, though.
- mustache_kimono 4y ago
  There are a number of python utilities (via scipy, etc.) that claim to do this by checking for the frequency cutoff. See FLAD and fakeflac.
  - joveian 4y ago
    I couldn't find FLAD but fakeflac doesn't seem to work, even given actual mp3s it claims they are fine :(. I'm guessing sometimes frequency cutoff is due to using less than ideal microphones, although not the extreme cutoffs (I'm fairly sure it would be gradual in that case). It also seems there is often a fair amount of noise around 20kHz that I'm guessing is from recording equipment, sometimes on tracks that otherwise look suspiciously like they are from lossy audio. Additionally, in some cases there could potentially be high frequency cutoffs in processing while otherwise being lossless, particularly stuff like electronic music where there can be a lot of processing in general. So I'm not sure if it is necessarily all that easy to tell in general, although there are certainly some from Bandcamp that seem highly likely to be conversions from lossy formats :(.
- gsora 4y ago
  You can read the spectrogram. https://interviewfor.red https://interviewfor.red has lots of info about that exact thing, because in some uh… very peculiar communities, sharing a lossy file passed as a lossless file is a serious offense.
  - piperswe 4y ago
    I love how that site has become the de facto main introduction resource to audio compression - makes sense though!
    - gsora 4y ago
      It’s the best layman resource out there IMO, also the fact that you might get a chance to get into RED… nothing wrong in that :-)
  - aaaaaaaaaaab 4y ago
    I see you’re a man of culture ( ͡° ͜ʖ ͡°)
  - crazygringo 4y ago
    Yup, it's incredibly clear in the spectogram. You're basically looking for two things: first, an abrupt stop to frequencies above a certain point -- compression uses applies a low-pass filter to eliminate them. Second, a bunch of rectangular-ish boxes where signal is missing. The spectrogram will look like swiss cheese in a way -- this is where the compression algorithm decides audio information can be deleted because it's "masked" by e.g. louder surrounding sounds. (Generally speaking, the more aggressive the compression, the lower the frequency ceiling, and the more/bigger swiss cheese holes.) On the other hand, if you run e.g. AAC compression at a very high bitrate and explicitly force no lowpass filter at all (which virtually nobody does in practice, but you can do from a command-line tool)... you won't be able to see any qualitative difference. Because you won't be able to hear any qualitative difference either. :)
- TonyTrapp 4y ago
  Audiochecker (https://archive.org/details/Audiochecker.v2.0.Beta.Build.457 https://archive.org/details/Audiochecker.v2.0.Beta.Build.457) is amazingly good at detecting WAV/FLAC that is really just a re-encoded from MPEG. It's specific to MPEG though, so it wouldn't be able to detect e.g. Vorbis converted to FLAC.
  - pabs3 4y ago
    Looks like that uses aucdtect: http://tausoft.org/wiki/aucdtect http://tausoft.org/wiki/aucdtect I wonder if there is any open source implementation of the algorithm it uses. http://tausoft.org/wiki/True_Audio_Checker_Algorithm http://tausoft.org/wiki/True_Audio_Checker_Algorithm
  - deleted 4y ago
    [deleted]
- dTal 4y ago
  There is https://losslessaudiochecker.com/ https://losslessaudiochecker.com/ - not sure how well it works.
aaaaaaaaaaab 4y ago
Yep, that's a smart feature. It had actually uncovered bugs in FFmpeg's FLAC encoder multiple times. [1][2] (why they didn't just use the reference implementation is beyond me...) But I have one nitpick with FLAC in this regard: they chose MD5 as the checksum instead of something sensible like CRC32/CRC64... It makes no sense, because we're not doing cryptography here - we're doing an integrity check. Moreover it makes a parallelized FLAC encoder somewhat problematic to implement, as there's always going to be a serial bottleneck at the end for computing the MD5 hash. CRC on the other hand could easily be parallelized. But I'm afraid we'll have to live with this shortcoming forever now ¯\_(ツ)_/¯ [1] https://trac.ffmpeg.org/ticket/4628 https://trac.ffmpeg.org/ticket/4628 [2] https://trac.ffmpeg.org/ticket/4421 https://trac.ffmpeg.org/ticket/4421
- mustache_kimono 4y ago
  Agreed. FYI, ffmpeg's hash testing format [0], and therefore my utility dano, allows the use of CRC32, etc., though. > But I'm afraid we'll have to live with this shortcoming forever now ¯\_(ツ)_/¯ I don't believe there is a fundamental reason why a later version of FLAC couldn't/wouldn't allow different algos as I think the raw decoded bitstream is checksum-ed and is just stored as a FLAC tag[1]. So they could just version that tag. [0]: http://underpop.online.fr/f/ffmpeg/help/hash.htm.gz http://underpop.online.fr/f/ffmpeg/help/hash.htm.gz [1]: Language corrected. See comment.
  - aaaaaaaaaaab 4y ago
    >I think the file is decoded and then checksum-ed and the checksum is just stored as a FLAC tag. Sure, it's just metadata. It's just about backwards compatibility. By the way, the checksum is calculated on the raw data fed into the encoder. It doesn't make sense to decode the data you've just encoded.
    - crtasm 4y ago
      > It doesn't make sense to decode the data you've just encoded. Unless you're also writing to disk and reading it back before decoding - I'm not sure which tool you're referring to.
samstave 4y ago
ELI5: What is preserved when it is "LOSSLESS" - what portion of a wave, or quality or (i dont know what term to ask) is dropped when "LOSS" occurs?
- fxtentacle 4y ago
  usually frequencies too high to be heard.
  - fuckstick 4y ago
    The lossless encoded signal is bandwidth limited already - lossy compression has to exploit more sophisticated perceptual phenomena such as masking. Cutting down the bandwidth (lowering the sample rate) wont sound as good as an actual lossy codec - otherwise we could just resample.
    - samstave 4y ago
      #Subscribe
- franga2000 4y ago
  From my understanding (I took two relevant undergrad courses, definitely not an expert) lossy compression can involve either losing certain frequencies (like those above/below human hearing), or losing the accuracy of the reproduction (like a certain frequency component of the sound could be the smallest bit louder/quieter or higher/lower than it was originally).
- CamperBob2 4y ago
  The key point is masking: if a sound at a given frequency is loud enough, you are less likely to perceive weaker sounds at other frequencies. So there is little point in wasting bits on those frequencies during the time intervals where they are being swamped. The exact frequency/amplitude relationships where masking effects come into play were studied by the telecoms early on (meaning in the 1950s-1960s era), and are still a key part of most lossy encoding models these days. See https://en.wikipedia.org/wiki/Critical_band https://en.wikipedia.org/wiki/Critical_band and https://en.wikipedia.org/wiki/Psychoacoustics https://en.wikipedia.org/wiki/Psychoacoustics as an introduction to the principles.
Lammy 4y ago
This was how I discovered my problem when I had bad non-ECC RAM in a machine with ZFS.
- valbu 4y ago
  Not often do I find a comment that brings up very similar experience. Had intermittent data corruption because memory bit error on very upper range that almost always was unused and went unnotced a long time. ZFS backend PC was actually ok, it was main PC that used the share but point remains. After that no more non-ECC memory ever on any computer for me (ok except some laptops).
  - joecool1029 4y ago
    I'm only going to add a related anecdote that wasn't a failing of ECC vs non-ECC but rather of BIOS behavior. Background: Lenovo Thinkpad T520 laptop, random crashes and data corruption. Diagnosis: Eventually let memtestx86+ run a bunch of times for like a week and it wasn't showing any errors. Finally about to give up I pressed some key on the keyboard and it blew errors immediately all over the screen. This suggested EC or maybe some BIOS-controlled keyboard driver was writing to memory it shouldn't have been. Fix: I am a Linux user, the kernel has an option to reserve low memory for poorly behaving BIOS that likes to write where it shouldn't. CONFIG_X86_RESERVE_LOW should be set to at least 64kb and increased up to 640kb if this issues continue to happen. There are some other options to scan for this misbehavior but I honestly don't know how Linux currently handles it: https://lkml.org/lkml/2013/11/11/683 https://lkml.org/lkml/2013/11/11/683
    - TD-Linux 4y ago
      Linux actually switched recently to leaving the whole bottom 1MB to the BIOS: https://lore.kernel.org/lkml/YLx%2FiA8xeRzwhXJn@zn.tnic/T/#u https://lore.kernel.org/lkml/YLx%2FiA8xeRzwhXJn@zn.tnic/T/#u Apparently Windows now does it too because too many BIOSes are buggy: https://bugzilla.kernel.org/show_bug.cgi?id=16661#c2 https://bugzilla.kernel.org/show_bug.cgi?id=16661#c2
jhallenworld 4y ago
Actually I have a related question about CDs: do any CD players indicate when the (Reed-Solomon I think) forward error correction actually corrects any errors? This could give an indication of the quality of the media. It could be total corrected error count, or corrected error count per minute.. something like that. Second, do any give an indication of when the error correction fails: I think CD players just fill the missing data with the last sample value (certainly this was the case for first gen CD players)- but this FLAC encoding gives a better procedure: replace the missing samples from those of a predictive model. But either way, it would be nice to know when the playback is not perfect.
- hansjorg 4y ago
  As to your second question: > Some ROM drives are capable of reporting C2 error information along with the audio data and some ripping software can use this information to determine whether the retrieved audio data is valid or not. A standardised mechanism for ROM drives to report C2 error information is documented in the Multi-Media Command (MMC) standard https://docs.linn.co.uk/wiki/index.php/CD_Ripping_Terminology https://docs.linn.co.uk/wiki/index.php/CD_Ripping_Terminolog...
- Lammy 4y ago
  I've never personally seen that on a dedicated player (appliance nor software), I assume because they are more interested in keeping the music playing. Exact Audio Copy shows it but will spend minutes re-reading the same frames: https://i.imgur.com/ZGozdhz.png https://i.imgur.com/ZGozdhz.png (screenshot mine, unfortunately)
  - aaaaaaaaaaab 4y ago
    If you’re somewhat lucky, you’ll be able to repair it with CueTools via CTDB.
- stevenwoo 4y ago
  My experience backing up up audio cds with Exact Audio Copy tells me that CD players have limited error correction facilities and using EAC is the only way to know! (read retail CD brand new get same as reference checksum, read same CD ten years later, mismatch errors popup on a few tracks), the FAQ on EAC gives some details on CD player capabilities - standalone audio CD players use oversampling and more but most don't tell you when it kicks in - there's typically only one small LCD/LED line of display. https://www.exactaudiocopy.de/en/index.php/support/faq/ https://www.exactaudiocopy.de/en/index.php/support/faq/
wongarsu 4y ago
PNG - another very well engineered format - also stores a checksum of the image data as well as checksums of every metadata block. But I guess most people and thus file formats don't care. If your file's corrupted and you don't notice it doesn't matter. And if you do notice you don't need a checksum to tell you. I don't necessarily agree (knowing where the corruption occurred is useful), but I can see the reasoning.
PaulDavisThe1st 4y ago
What would the point be of storing a checksum of the original data if the compression technique is known to be lossy?
- wyldfire 4y ago
  You could use a CRC to verify metadata contents. But IMO these are the properties of a good container file / stream format. FLAC might define its own but IMO most codecs could work well with a generic container.
- mustache_kimono 4y ago
  It doesn't? However, it may be significant to store a checksum of the lossy bitstream?
giantrobot 4y ago
Checksumming in media container formats is tricky because different media formats have different features. So each codec and combination of track/codec needs its own special case on how and when a checksum is performed. In the case of FLAC the situation is straight forward, you've got uncompressed audio and a codec that's performing lossless compression. You want to know what was encoded losslessly matches the input. It's no different from a checksum in a compressed archive. It's less straightforward with a lossy compressed codec. Are we just checksumming the compressed bitstream? Where in the handling of the bitstream are we doing the checksum? In MOV/MP4 a media track can have arbitrary start and end times. It makes doing in/out cuts super fast. But when it comes to checksumming the track do we checksum all the samples in the track or only the samples that will play? If a file if flattened and a track has all the samples out of the playable region discarded do we now have to recalculate the checksum? Media tracks can also have header tags that indicate stuff like display color space, displayed size, or even track level metadata. Are we checksumming the track bitstream and important track headers? A video needing to render in a particular color space will be materially affected if it's bitstream is preserved but color space tags are dropped. If you want to hack in checksums for track bitstreams both MOV/MP4 and Matroska can accept arbitrary tags for tracks so you can just write a checksum to a track header tag if you really want to. You really need a table of rules for doing the checksum otherwise it's a wasted effort.
comprev 4y ago
I have a fairly large collection of DJ music and this tool sounds extremely helpful for management. Thank you!
gsich 4y ago
You probably can store hashes in MKV, not sure if there is a dedicated field for that. I want something like that for JPG, with parity.
deleted 4y ago
[deleted]

lake_vincent 4y ago

What are the reasons why not? What are the obstacles/drawbacks to the FLAC approach? I have no domain knowledge here, just curious

lazide 4y ago
For the most part, no one cares to do so is why. Hell, 99% of all computers are still running on non-checksummed filesystems, if not 99.99%. For the most part, only a tiny minority of people are even aware of bit rot, let alone have anything they simultaneously care enough about to protect against, and have enough ownership of it to attempt to try.

hnuser123456 4y ago

Are there any good ways to tell if audio has been lossy compressed but then re-encoded in a lossless format, or deceptively high lossy bitrate?

dmitri_ignat 4y ago
You can view it with a spectrogram, audio which has been lossily encoded will have a telltale cutoff in the high frequencies. The lossier the encoding, the lower the frequency ceiling will be.
- sdenton4 4y ago
  Another thing to look for is deleted or very flat energy between harmonics. Reducing bit rate in inter harmonic regions is most of what's meant by 'psychoacoustics.' I imagine low pass filtering depends a bit on which compression you're using. None of these tricks will work for detecting GAN compressed audio, though.
mustache_kimono 4y ago
There are a number of python utilities (via scipy, etc.) that claim to do this by checking for the frequency cutoff. See FLAD and fakeflac.
- joveian 4y ago
  I couldn't find FLAD but fakeflac doesn't seem to work, even given actual mp3s it claims they are fine :(. I'm guessing sometimes frequency cutoff is due to using less than ideal microphones, although not the extreme cutoffs (I'm fairly sure it would be gradual in that case). It also seems there is often a fair amount of noise around 20kHz that I'm guessing is from recording equipment, sometimes on tracks that otherwise look suspiciously like they are from lossy audio. Additionally, in some cases there could potentially be high frequency cutoffs in processing while otherwise being lossless, particularly stuff like electronic music where there can be a lot of processing in general. So I'm not sure if it is necessarily all that easy to tell in general, although there are certainly some from Bandcamp that seem highly likely to be conversions from lossy formats :(.
gsora 4y ago
You can read the spectrogram. https://interviewfor.red https://interviewfor.red has lots of info about that exact thing, because in some uh… very peculiar communities, sharing a lossy file passed as a lossless file is a serious offense.
- piperswe 4y ago
  I love how that site has become the de facto main introduction resource to audio compression - makes sense though!
  - gsora 4y ago
    It’s the best layman resource out there IMO, also the fact that you might get a chance to get into RED… nothing wrong in that :-)
- aaaaaaaaaaab 4y ago
  I see you’re a man of culture ( ͡° ͜ʖ ͡°)
- crazygringo 4y ago
  Yup, it's incredibly clear in the spectogram. You're basically looking for two things: first, an abrupt stop to frequencies above a certain point -- compression uses applies a low-pass filter to eliminate them. Second, a bunch of rectangular-ish boxes where signal is missing. The spectrogram will look like swiss cheese in a way -- this is where the compression algorithm decides audio information can be deleted because it's "masked" by e.g. louder surrounding sounds. (Generally speaking, the more aggressive the compression, the lower the frequency ceiling, and the more/bigger swiss cheese holes.) On the other hand, if you run e.g. AAC compression at a very high bitrate and explicitly force no lowpass filter at all (which virtually nobody does in practice, but you can do from a command-line tool)... you won't be able to see any qualitative difference. Because you won't be able to hear any qualitative difference either. :)
TonyTrapp 4y ago
Audiochecker (https://archive.org/details/Audiochecker.v2.0.Beta.Build.457 https://archive.org/details/Audiochecker.v2.0.Beta.Build.457) is amazingly good at detecting WAV/FLAC that is really just a re-encoded from MPEG. It's specific to MPEG though, so it wouldn't be able to detect e.g. Vorbis converted to FLAC.
- pabs3 4y ago
  Looks like that uses aucdtect: http://tausoft.org/wiki/aucdtect http://tausoft.org/wiki/aucdtect I wonder if there is any open source implementation of the algorithm it uses. http://tausoft.org/wiki/True_Audio_Checker_Algorithm http://tausoft.org/wiki/True_Audio_Checker_Algorithm
- deleted 4y ago
  [deleted]
dTal 4y ago
There is https://losslessaudiochecker.com/ https://losslessaudiochecker.com/ - not sure how well it works.

aaaaaaaaaaab 4y ago

Yep, that's a smart feature. It had actually uncovered bugs in FFmpeg's FLAC encoder multiple times. [1][2] (why they didn't just use the reference implementation is beyond me...) But I have one nitpick with FLAC in this regard: they chose MD5 as the checksum instead of something sensible like CRC32/CRC64... It makes no sense, because we're not doing cryptography here - we're doing an integrity check. Moreover it makes a parallelized FLAC encoder somewhat problematic to implement, as there's always going to be a serial bottleneck at the end for computing the MD5 hash. CRC on the other hand could easily be parallelized. But I'm afraid we'll have to live with this shortcoming forever now ¯\_(ツ)_/¯ [1] https://trac.ffmpeg.org/ticket/4628 https://trac.ffmpeg.org/ticket/4628 [2] https://trac.ffmpeg.org/ticket/4421 https://trac.ffmpeg.org/ticket/4421

mustache_kimono 4y ago
Agreed. FYI, ffmpeg's hash testing format [0], and therefore my utility dano, allows the use of CRC32, etc., though. > But I'm afraid we'll have to live with this shortcoming forever now ¯\_(ツ)_/¯ I don't believe there is a fundamental reason why a later version of FLAC couldn't/wouldn't allow different algos as I think the raw decoded bitstream is checksum-ed and is just stored as a FLAC tag[1]. So they could just version that tag. [0]: http://underpop.online.fr/f/ffmpeg/help/hash.htm.gz http://underpop.online.fr/f/ffmpeg/help/hash.htm.gz [1]: Language corrected. See comment.
- aaaaaaaaaaab 4y ago
  >I think the file is decoded and then checksum-ed and the checksum is just stored as a FLAC tag. Sure, it's just metadata. It's just about backwards compatibility. By the way, the checksum is calculated on the raw data fed into the encoder. It doesn't make sense to decode the data you've just encoded.
  - crtasm 4y ago
    > It doesn't make sense to decode the data you've just encoded. Unless you're also writing to disk and reading it back before decoding - I'm not sure which tool you're referring to.

samstave 4y ago

ELI5: What is preserved when it is "LOSSLESS" - what portion of a wave, or quality or (i dont know what term to ask) is dropped when "LOSS" occurs?

fxtentacle 4y ago
usually frequencies too high to be heard.
- fuckstick 4y ago
  The lossless encoded signal is bandwidth limited already - lossy compression has to exploit more sophisticated perceptual phenomena such as masking. Cutting down the bandwidth (lowering the sample rate) wont sound as good as an actual lossy codec - otherwise we could just resample.
  - samstave 4y ago
    #Subscribe
franga2000 4y ago
From my understanding (I took two relevant undergrad courses, definitely not an expert) lossy compression can involve either losing certain frequencies (like those above/below human hearing), or losing the accuracy of the reproduction (like a certain frequency component of the sound could be the smallest bit louder/quieter or higher/lower than it was originally).
CamperBob2 4y ago
The key point is masking: if a sound at a given frequency is loud enough, you are less likely to perceive weaker sounds at other frequencies. So there is little point in wasting bits on those frequencies during the time intervals where they are being swamped. The exact frequency/amplitude relationships where masking effects come into play were studied by the telecoms early on (meaning in the 1950s-1960s era), and are still a key part of most lossy encoding models these days. See https://en.wikipedia.org/wiki/Critical_band https://en.wikipedia.org/wiki/Critical_band and https://en.wikipedia.org/wiki/Psychoacoustics https://en.wikipedia.org/wiki/Psychoacoustics as an introduction to the principles.

Lammy 4y ago

This was how I discovered my problem when I had bad non-ECC RAM in a machine with ZFS.

valbu 4y ago
Not often do I find a comment that brings up very similar experience. Had intermittent data corruption because memory bit error on very upper range that almost always was unused and went unnotced a long time. ZFS backend PC was actually ok, it was main PC that used the share but point remains. After that no more non-ECC memory ever on any computer for me (ok except some laptops).
- joecool1029 4y ago
  I'm only going to add a related anecdote that wasn't a failing of ECC vs non-ECC but rather of BIOS behavior. Background: Lenovo Thinkpad T520 laptop, random crashes and data corruption. Diagnosis: Eventually let memtestx86+ run a bunch of times for like a week and it wasn't showing any errors. Finally about to give up I pressed some key on the keyboard and it blew errors immediately all over the screen. This suggested EC or maybe some BIOS-controlled keyboard driver was writing to memory it shouldn't have been. Fix: I am a Linux user, the kernel has an option to reserve low memory for poorly behaving BIOS that likes to write where it shouldn't. CONFIG_X86_RESERVE_LOW should be set to at least 64kb and increased up to 640kb if this issues continue to happen. There are some other options to scan for this misbehavior but I honestly don't know how Linux currently handles it: https://lkml.org/lkml/2013/11/11/683 https://lkml.org/lkml/2013/11/11/683
  - TD-Linux 4y ago
    Linux actually switched recently to leaving the whole bottom 1MB to the BIOS: https://lore.kernel.org/lkml/YLx%2FiA8xeRzwhXJn@zn.tnic/T/#u https://lore.kernel.org/lkml/YLx%2FiA8xeRzwhXJn@zn.tnic/T/#u Apparently Windows now does it too because too many BIOSes are buggy: https://bugzilla.kernel.org/show_bug.cgi?id=16661#c2 https://bugzilla.kernel.org/show_bug.cgi?id=16661#c2

jhallenworld 4y ago

Actually I have a related question about CDs: do any CD players indicate when the (Reed-Solomon I think) forward error correction actually corrects any errors? This could give an indication of the quality of the media. It could be total corrected error count, or corrected error count per minute.. something like that. Second, do any give an indication of when the error correction fails: I think CD players just fill the missing data with the last sample value (certainly this was the case for first gen CD players)- but this FLAC encoding gives a better procedure: replace the missing samples from those of a predictive model. But either way, it would be nice to know when the playback is not perfect.

hansjorg 4y ago
As to your second question: > Some ROM drives are capable of reporting C2 error information along with the audio data and some ripping software can use this information to determine whether the retrieved audio data is valid or not. A standardised mechanism for ROM drives to report C2 error information is documented in the Multi-Media Command (MMC) standard https://docs.linn.co.uk/wiki/index.php/CD_Ripping_Terminology https://docs.linn.co.uk/wiki/index.php/CD_Ripping_Terminolog...
Lammy 4y ago
I've never personally seen that on a dedicated player (appliance nor software), I assume because they are more interested in keeping the music playing. Exact Audio Copy shows it but will spend minutes re-reading the same frames: https://i.imgur.com/ZGozdhz.png https://i.imgur.com/ZGozdhz.png (screenshot mine, unfortunately)
- aaaaaaaaaaab 4y ago
  If you’re somewhat lucky, you’ll be able to repair it with CueTools via CTDB.
stevenwoo 4y ago
My experience backing up up audio cds with Exact Audio Copy tells me that CD players have limited error correction facilities and using EAC is the only way to know! (read retail CD brand new get same as reference checksum, read same CD ten years later, mismatch errors popup on a few tracks), the FAQ on EAC gives some details on CD player capabilities - standalone audio CD players use oversampling and more but most don't tell you when it kicks in - there's typically only one small LCD/LED line of display. https://www.exactaudiocopy.de/en/index.php/support/faq/ https://www.exactaudiocopy.de/en/index.php/support/faq/

wongarsu 4y ago

PNG - another very well engineered format - also stores a checksum of the image data as well as checksums of every metadata block. But I guess most people and thus file formats don't care. If your file's corrupted and you don't notice it doesn't matter. And if you do notice you don't need a checksum to tell you. I don't necessarily agree (knowing where the corruption occurred is useful), but I can see the reasoning.

PaulDavisThe1st 4y ago

What would the point be of storing a checksum of the original data if the compression technique is known to be lossy?

wyldfire 4y ago
You could use a CRC to verify metadata contents. But IMO these are the properties of a good container file / stream format. FLAC might define its own but IMO most codecs could work well with a generic container.
mustache_kimono 4y ago
It doesn't? However, it may be significant to store a checksum of the lossy bitstream?

giantrobot 4y ago

Checksumming in media container formats is tricky because different media formats have different features. So each codec and combination of track/codec needs its own special case on how and when a checksum is performed. In the case of FLAC the situation is straight forward, you've got uncompressed audio and a codec that's performing lossless compression. You want to know what was encoded losslessly matches the input. It's no different from a checksum in a compressed archive. It's less straightforward with a lossy compressed codec. Are we just checksumming the compressed bitstream? Where in the handling of the bitstream are we doing the checksum? In MOV/MP4 a media track can have arbitrary start and end times. It makes doing in/out cuts super fast. But when it comes to checksumming the track do we checksum all the samples in the track or only the samples that will play? If a file if flattened and a track has all the samples out of the playable region discarded do we now have to recalculate the checksum? Media tracks can also have header tags that indicate stuff like display color space, displayed size, or even track level metadata. Are we checksumming the track bitstream and important track headers? A video needing to render in a particular color space will be materially affected if it's bitstream is preserved but color space tags are dropped. If you want to hack in checksums for track bitstreams both MOV/MP4 and Matroska can accept arbitrary tags for tracks so you can just write a checksum to a track header tag if you really want to. You really need a table of rules for doing the checksum otherwise it's a wasted effort.

comprev 4y ago

I have a fairly large collection of DJ music and this tool sounds extremely helpful for management. Thank you!

gsich 4y ago

You probably can store hashes in MKV, not sure if there is a dedicated field for that. I want something like that for JPG, with parity.

deleted 4y ago

[deleted]