63 ms·

FLAC 1.4.0 released – added support for 32-bit audio

171 points by thrdbndndn 4y ago

theandrewbailey 4y ago
> Compression of preset -3 through -8 was slightly improved at the cost of a small decrease in encoding speed by increasing the precision with which autocorrelation was calculated (Martijn van Beurden) Would love to see some benchmarks on this. Since FLAC hit 1.0 over 20 years ago, CPU speed has increased by ~100x, so a small hit on encode speed for better compression mostly doesn't matter. Then again, storage space has increased by the same amount, too.
- mmozeiko 4y ago
  Btw there are alternative flac encoders, like FLACCL using GPU: http://cue.tools/wiki/FLACCL http://cue.tools/wiki/FLACCL It compresses much faster than software libflac and gives smaller output files.
michaelwww 4y ago
After reading "How Music Works" by David Byrne, it seem like FLAC is solving the wrong problem. No one needs over 4 billion discrete levels.
- _joel 4y ago
  Nyquist Shannon theorem, sure. There are situations where you'd use 32bit audio but not for mastered music you'd listen to at home really. Although people still won't believe you, hence the market for thousand dollar cables and other snake oil.
  - Night_Thastus 4y ago
    From my understanding, doesn't Nyquist-Shannon assume you take infinite steps to refine the discrete steps into the final waveform? Since real-world DACs don't have infinite taps, either increasing the number of samples per second in the original audio or the number of steps done by the filter will improve how close it gets to the original waveform. Would that apply to the number of bits used to represent the level as well? I thought that was mainly useful to give some additional headroom when editing? (Obviously there's diminishing returns either way)
    - colejohnson66 4y ago
      Nyquist-Shannon simply says that you need 2x the sample rate of the highest frequency to perfectly recreate something. So, to recreate a 20 kHz wave that was sampled at 16 bits, you'd need 40 kHz sample rate. Bump that up to 22.05 kHz (for anti-aliasing) and you get CD audio. The bit depth, OTOH, is your signal-noise ratio (noise floor) when sampling/quantizing said audio. It's unrelated to Nyquist-Shannon. More bits gives you less noise when reproducing it. For CD audio, 16 bits was chosen to give an acceptable SNR of 96 dB. Monty Montgomery (of xipf.org) has a nice YouTube video related to this: https://www.youtube.com/watch?v=cIQ9IXSUzuM https://www.youtube.com/watch?v=cIQ9IXSUzuM
      - Night_Thastus 4y ago
        Yes, you only need that many samples given you know a maximum frequency. But to actually perform the conversion from discrete samples to continuous wave takes infinitely many steps. The number a given DAC actually does varies. Some do hundreds, some thousands, a couple rare (very expensive) ones do 100k or more. That's what I'm saying. Increase the number of samples in the audio itself, and the DAC can get closer to the original.
        colejohnson66 4y ago
        Except it's not. The video shows that. In the real world, the DAC doesn't immediately jump from code N to N+1; it slides (the "slew rate" of the opamp). More bits just makes the difference between codes N and N+1 smaller; i.e. where it slides to is more accurate to the original. The "stair step" or "straight line" view you get from programs like Audacity is a lie.
        jrmcauliffe 4y ago
        A final low pass filter (tuned to remove anything above half the sample rate) allows the accurate reconstruction of the original analogue waveform from the sampled digital data. ( The vertical 'stair step' of sampled data is very high frequency information which gets removed, 'bending' the output back into it's original shape). The video linked above is well worth a watch.
- atoav 4y ago
  I am a recording engineer and I will choose 32 bits over 24 any day when recording. Of course you need the right (exceptionally good!) preamps to even make a difference here, but when you are manipulating audio 32 bits and upwards (for summing) are the default (for a reason). Having something to store those files with a little bit of lossless compression is a welcome addition
  - michaelwww 4y ago
    > I will choose 32 bits over 24 any day why?
    - Blueskytech 4y ago
      Because if the recording goes past the clipping range for digital audio it still records the data and preserves it. 32bit recording has serious advantages because it’s much easier to record a hot input because digital clipping is not possible. You just need to pull the volume down on the recorded clip after the fact and there is no clipping.
  - squeaky-clean 4y ago
    It's very unlikely your audio interface can actually record at 32 bit resolution. https://www.mojo-audio.com/blog/the-24bit-delusion/ https://www.mojo-audio.com/blog/the-24bit-delusion/ 32bit or more makes sense for dsp and summing, and non-musical data storage. But your I/O is most likely not giving 32 bits
    - jrajav 4y ago
      32-bit is common in field recordings, like with a Zoom (and there's reason to believe they can legitimately take advantage of it).
      - deleted 4y ago
        [deleted]
      - squeaky-clean 4y ago
- colejohnson66 4y ago
  You're correct; lossless formats aren't needed by the average consumer. But the problem they solve is a very real thing. Think of the difference between a repeatedly edited JPG vs. a repeatedly edited PNG. Every save of the JPG destroys information irretrievably, whereas every save of the PNG keeps it all. It's the same with MP3 vs. FLAC: MP3 throws away information each export, but FLAC doesn't. For an example, music artists/producers can't afford to lose information when they "bounce" a track. FLAC solves that. "People" don't need lossless, but for those that do, it's great.
  - pwiecz 4y ago
    "People" don't need just lossless, they need The Beatles to release all of their pre-master tracks to be released in lossless format, so they of them can create their own remastered versions of a the tracks.
  - michaelwww 4y ago
    16 bit is not enough?
    - deleted 4y ago
      [deleted]
    - pavon 4y ago
      No, if you want your final output to be 16-bit (eg CD quality), then you need more bits to work with when mixing. Each operation you perform will be requantized (rounded) to 16-bit resolution adding a bit of noise each time, until eventually the lower 4 bits (for example) are nothing but noise, and your signal is effectively only 12 bits. One the other hand if you did that mixing at 24-bit resolution, then you still have 20 bits of good data remaining, you throw away the bottom 4, and have nice clean 16-bit data to distribute. 32-bit integer is overkill in most situations, which is why it wasn't implemented for so long. 32-bit float is used a lot because it has 24-bits of resolution, but also has a huge scaling range, so you don't have to worry about clipping during intermediate operations, just at the end.
  - squeaky-clean 4y ago
- adgjlsfhk1 4y ago
  There are definitely scientific uses that might want to (ab)use FLAC to store sound-like data that would benefit from 32 bit.
- mmastrac 4y ago
  There's a real use case for storing original tracks and masters in 32-bit and down-sampling for consumer use.
- tombert 4y ago
  While I doubt that human ears need 4 billion discrete levels, I think it's important for archival purposes.
  - klodolph 4y ago
    Is it? What is your reasoning? And why would you pick 32-bit, instead of 24-bit or 64-bit or something else? Actual audio equipment has a noise floor. The encoding depth you use also has a noise floor. If the noise floor of the encoding is far below the noise floor of the signal, then it won't be perceptible. So you choose to put your noise floor at some amount below the existing noise floor. Not infinitely below, because that would require infinite bits. This is the reasoning you'd use for 24-bit audio, which has a very comfortable noise floor of -144 dB, which leaves a large margin even for extreme low-noise professional equipment (you might see -120 dB ish for extremely good equipment). At 24-bit, even passive components like transformers and resistors are contributing measurable amounts of noise. The reason you might pick 32-bit in practice is so that you can have lots of headroom for some DSP algorithm, or do lots of sums of different signals without accumulating quantization error. The final "archive" file will still have worse than 24-bit precision.
    - tombert 4y ago
      Fair enough! That's what I get for commenting on something I'm not as familiar enough with as I should be.
- matheusmoreira 4y ago
  How do you know no one needs it for anything? How do you know no one will ever need it in the future? It's not possible to know that. All we can do is capture the real world data with as much quality and precision as possible.
- jlouis 4y ago
  I don't think it's for storing music.
- navjack27 4y ago
  I got this one "remaster" of an album that I like that I've been working on that the 32-bit version that 100% definitely goes over 0 DB just happens to sound a whole lot better when I don't correct for that so on occasion I like to load up that album in Ableton live and listen to it exactly like that.
pavon 4y ago
Nice! I (ab)use FLAC to store waveform data that isn't audio, but compresses better using audio codecs than with general purpose compression algorithms. This will let me remove some hacks to compress 32-bit data as multiple channels.
- colejohnson66 4y ago
  What data are you compressing?
  - nimish 4y ago
    I've used it to record oscilloscope captures. Especially useful since audio cards are cheap high res adc's.
    - mywittyname 4y ago
      I'm amazed that consumer-grade audio cards are still available and cheap, to be honest. I can't recall having seen one in years, then again, I haven't had a reason to look for them in years either.
      - wheels 4y ago
        To be clear: actual "PCI-slot"-ish audio cards are now rare. But USB / Thunderbolt (and previously Firewire) external interfaces are a dime a dozen. Serious audio mostly uses them now.
      - baybal2 4y ago
        Microchips are cheap, once initial investment is made. I guess, it only costs ~10¢ for new SoundBlasters at 90nm-130nm. The price of 10 sound cards will be enough to pay for a whole wafer.
      - zymhan 4y ago
        Creative is still marketing them to gamers, amazingly. Can't say I haven't been tempted.
Night_Thastus 4y ago
I had no idea FLAC didn't store 32-bit. I'll need to look over my library and make sure nothing strange is going on. I'm curious if sites like Qobuz will suddenly replace some of their existing files with higher-bit copies if they already had them and just had no way to encode them as FLAC.
- klodolph 4y ago
  It is extremely hard to justify 32-bit audio from a technical standpoint, since it requires that every component in your audio chain has better than 144 dB dynamic range. In practice, just about none of the analog components in your audio chain will have that kind of dynamic range, and any analog components in the recording chain are unlikely to have that range either. The kind of equipment I'd expect to see in order to record that kind of range is where you'd have something like multiple microphones, and switch back and forth between them based on the level of the material you're recording (which would have to be something like actual gunfire or explosions). I don't know what kind of equipment you'd need to reproduce this dynamic range, but I don't think it's audio equipment. My guess is that the main real application of this is going to be compression of non-audio data that happens to compress well with FLAC.
  - simcop2387 4y ago
    I could see it as part of a pipeline/workflow where having that dynamic range lets you not worry about losing meaningful information through everything, even if you're going to compress it and take it back to 16bit cd quality at the end. Being able to losslessly store the results of each step in a FLAC file should be better than raw data since it'll be compressed and easier to manage then. That said, at the edges of that I totally agree even 24-bit can be questionable there since a lot of the analog side doesn't have a noise floor that would let it be meaningfully used.
    - klodolph 4y ago
      > That said, at the edges of that I totally agree even 24-bit can be questionable there since a lot of the analog side doesn't have a noise floor that would let it be meaningfully used. It's not that hard to beat 16-bit, which is 96 dB, using easy-to-find, off-the-shelf equipment. One example scenario is that you are recording something but you don't have a precise idea of how loud it will be ahead of time, so you record at low levels and rely on 24-bit capture to give you headroom above and noise floor below. Trying to capture at 16-bit can, in practice, be annoying and difficult because it is more likely that you will ruin takes by setting the gain wrong.
sprash 4y ago
Sadly still no support for variable blocksize.
- nayuki 4y ago
  I made my own encoder and experimented with variable block size (with dynamic programming) plus a massive amount of brute-force search for LPC parameters. But the amount of compression gain I could achieve over libflac was small and not worth the hours of encoding time. https://www.nayuki.io/page/benchmark-of-nayukis-flac-encoder https://www.nayuki.io/page/benchmark-of-nayukis-flac-encoder
  - sprash 4y ago
    Sometimes encoding time doesn't matter. Archiving is done once in a lifetime. I encode everything with "--lax -8Vepl32" which takes ages but since it runs as a low priority background task I don't care. Also, you can use heuristics to choose the right blocksize instead of brute-forcing it. If the algorithm is clever enough you might achieve slightly better results at almost zero costs.
    - freedom-fries 4y ago
      Interesting. Do you have numbers to show that variable block size can be done with "almost zero cost" as you said?
      - nayuki 4y ago
        I think "If the algorithm is clever enough" is a big weasel phrase just like https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler . Having studied and written many algorithms, I have to conservatively assume that a clever algorithm doesn't exist unless it is proven by construction.
LeoPanthera 4y ago
32-bit integer, not float. I didn't even know 32-bit integer was a thing. Float is far more common. (For audio.)
- survirtual 4y ago
  This does not matter… 32bits is 32bits is 32bits.
  - pavon 4y ago
    You are not going to get good compression reinterpreting a 32-bit float as if the bits were an integer.
    - survirtual 4y ago
      What? Most systems are 64bit now, I wouldn’t be doing signal processing on a system that wasn’t. If my signal has 32bits to work with during capture and my scratch space is 64bits during processing, I can assure you I am not losing any signal during conversion between int / float. What am I missing?
  - LeoPanthera 4y ago
    It absolutely does matter when the compression algorithm is expecting audio in a certain format. If you put something else in those bits you will get worse, or no, compression, and might as well just use zip.
    - survirtual 4y ago
      I must be missing something here, but last I checked 32bit float uses about 8bits for metadata —- positive / negative, exponent, etc. — leaving 24bits for signal storage. A 32bit uint can just store the bits of raw data and would allow for denser packing of bits. You can then do processing with a 64bit float after converting the int’s bits without loss. Pretty sure every ADC IC I looked at uses integers. During analog processing, using the term “float” is highly disingenuous for an analog waveform, and if an ADC spouts out an int, what float are we talking about here? I maybe have gone too deep in the weeds reading TI chip manuals and writing drivers for some adc chips for my 8 channel, 32bit, 192kHz recording system, and maybe I missed some nuances in what “32bit” means, but last I checked my audio is pretty solid even after converting the 32bit int to 64bit float during my signal processing. Audio engineering has been a massive pain because these people use different terms for basic things (great example is “phantom power” — it’s just a 48v, low-current power supply, but good luck finding a simple answer to that useless term) and there doesn’t seem to be any authoritative references to find answers, so I am sure I am missing something here.
thrdbndndn 4y ago
I personally can't care less about 32-bit, but unfortunately a few digital music stores I buy from use this format (delivered in WAV). And I don't want to downscale them simply for archival reasons. Now I can at least re-compress them losslessly in FLAC. (previously I have to use Wavpack, which isn't bad but I'd prefer to have a single format in my media library.)
- aaaaaaaaaaab 4y ago
  Boomkat?
  - thrdbndndn 4y ago
    I've heard of that too, but in my case it's batch of Japanese distributors like e-onkyo [1] or Ototoy [2]. [1] https://www.e-onkyo.com/music/ https://www.e-onkyo.com/music/ [2] https://ototoy.jp/top/ https://ototoy.jp/top/
    - astrange 4y ago
      Yes, unfortunately all of Japan has jumped headfirst into something called "hi-res audio" which is a complete scam, as CD audio is already the highest quality something mixed for humans could possibly need to be. I'd like to think it's an excuse to stop doing loudness war mastering, but it's probably just as bad. The real problem with CD audio is that it's premixed in stereo, and the real next step is Atmos/spatial audio. Just because you only have two ears doesn't mean you can't hear 360 degrees.
      - Maursault 4y ago
        > which is a complete scam I can hear the difference between 16-bit audio and 24-bit audio. 16-bit isn't bad at all, but 24-bit is definitely superior. I've worked with 32-bit on digital boards, and I think it is probably over-overkill, but whatever makes someone happy. But one thing I know for certain and beyond all possibility of doubt is that I only have two ears. All that is necessary for 3D sound is stereo speakers. To be clear, with only 2 speakers sound can be mixed to seem like it is coming from any direction the engineer desires, left, right, top, bottom, front, back, and everywhere in between. And stereo speakers are dead simple to set up correctly for 3D sound. Sure, you can use 6 or 8 or 10 speakers to get the same effect, if you can figure out how to properly set them up, which is not a given, but I think it's a dick-pull and a money grab because no one can hear with more than two ears. I realize there are Surround fanatics out there and in here that feel very strongly about it, but the physics of acoustics tells me their set ups are no better than and less efficient than stereo, and nearly always incorrectly configured.
aaaaaaaaaaab 4y ago
32bit float would be more interesting for music production.
ZoomZoomZoom 4y ago
Finally! I can ditch away 32b special casing in my project cleanup and backup automation! As much as I love WavPack[1] (it's great and has 32 bit integer/fp support since forever), Flac is much faster and probably more future-proof. Thanks a lot everyone at XIPH, your work really makes a difference. Edit: oops, I can't. No fp support yet. 1. https://www.wavpack.com/ https://www.wavpack.com/
- ksec 4y ago
  It is sad that a technically superior and patent free technology often lose out in the popularity contest. Sigh.
  - dTal 4y ago
    What do you mean? Both FLAC and WavPack claim to be patent-free.
    - ksec 4y ago
      I am not aware of FLAC claims to be patent free ( when it started ), it may be royalty free and patent free now given it has been 20+ years since 1.0. They do however claims they dont hold any patent on it. WavePack was specifically designed to be Patent Free, or using old and expired Patent techniques.
procodr 4y ago
And iTunes still does not support FLAC. .

171 points by thrdbndndn 4y ago

> Compression of preset -3 through -8 was slightly improved at the cost of a small decrease in encoding speed by increasing the precision with which autocorrelation was calculated (Martijn van Beurden) Would love to see some benchmarks on this. Since FLAC hit 1.0 over 20 years ago, CPU speed has increased by ~100x, so a small hit on encode speed for better compression mostly doesn't matter. Then again, storage space has increased by the same amount, too.

mmozeiko 4y ago
Btw there are alternative flac encoders, like FLACCL using GPU: http://cue.tools/wiki/FLACCL http://cue.tools/wiki/FLACCL It compresses much faster than software libflac and gives smaller output files.

michaelwww 4y ago

After reading "How Music Works" by David Byrne, it seem like FLAC is solving the wrong problem. No one needs over 4 billion discrete levels.

_joel 4y ago
Nyquist Shannon theorem, sure. There are situations where you'd use 32bit audio but not for mastered music you'd listen to at home really. Although people still won't believe you, hence the market for thousand dollar cables and other snake oil.
- Night_Thastus 4y ago
  From my understanding, doesn't Nyquist-Shannon assume you take infinite steps to refine the discrete steps into the final waveform? Since real-world DACs don't have infinite taps, either increasing the number of samples per second in the original audio or the number of steps done by the filter will improve how close it gets to the original waveform. Would that apply to the number of bits used to represent the level as well? I thought that was mainly useful to give some additional headroom when editing? (Obviously there's diminishing returns either way)
  - colejohnson66 4y ago
    Nyquist-Shannon simply says that you need 2x the sample rate of the highest frequency to perfectly recreate something. So, to recreate a 20 kHz wave that was sampled at 16 bits, you'd need 40 kHz sample rate. Bump that up to 22.05 kHz (for anti-aliasing) and you get CD audio. The bit depth, OTOH, is your signal-noise ratio (noise floor) when sampling/quantizing said audio. It's unrelated to Nyquist-Shannon. More bits gives you less noise when reproducing it. For CD audio, 16 bits was chosen to give an acceptable SNR of 96 dB. Monty Montgomery (of xipf.org) has a nice YouTube video related to this: https://www.youtube.com/watch?v=cIQ9IXSUzuM https://www.youtube.com/watch?v=cIQ9IXSUzuM
    - Night_Thastus 4y ago
      Yes, you only need that many samples given you know a maximum frequency. But to actually perform the conversion from discrete samples to continuous wave takes infinitely many steps. The number a given DAC actually does varies. Some do hundreds, some thousands, a couple rare (very expensive) ones do 100k or more. That's what I'm saying. Increase the number of samples in the audio itself, and the DAC can get closer to the original.
      - colejohnson66 4y ago
        Except it's not. The video shows that. In the real world, the DAC doesn't immediately jump from code N to N+1; it slides (the "slew rate" of the opamp). More bits just makes the difference between codes N and N+1 smaller; i.e. where it slides to is more accurate to the original. The "stair step" or "straight line" view you get from programs like Audacity is a lie.
        jrmcauliffe 4y ago
        A final low pass filter (tuned to remove anything above half the sample rate) allows the accurate reconstruction of the original analogue waveform from the sampled digital data. ( The vertical 'stair step' of sampled data is very high frequency information which gets removed, 'bending' the output back into it's original shape). The video linked above is well worth a watch.
atoav 4y ago
I am a recording engineer and I will choose 32 bits over 24 any day when recording. Of course you need the right (exceptionally good!) preamps to even make a difference here, but when you are manipulating audio 32 bits and upwards (for summing) are the default (for a reason). Having something to store those files with a little bit of lossless compression is a welcome addition
- michaelwww 4y ago
  > I will choose 32 bits over 24 any day why?
  - Blueskytech 4y ago
    Because if the recording goes past the clipping range for digital audio it still records the data and preserves it. 32bit recording has serious advantages because it’s much easier to record a hot input because digital clipping is not possible. You just need to pull the volume down on the recorded clip after the fact and there is no clipping.
- squeaky-clean 4y ago
  It's very unlikely your audio interface can actually record at 32 bit resolution. https://www.mojo-audio.com/blog/the-24bit-delusion/ https://www.mojo-audio.com/blog/the-24bit-delusion/ 32bit or more makes sense for dsp and summing, and non-musical data storage. But your I/O is most likely not giving 32 bits
  - jrajav 4y ago
    32-bit is common in field recordings, like with a Zoom (and there's reason to believe they can legitimately take advantage of it).
    - deleted 4y ago
      [deleted]
    - squeaky-clean 4y ago
colejohnson66 4y ago
You're correct; lossless formats aren't needed by the average consumer. But the problem they solve is a very real thing. Think of the difference between a repeatedly edited JPG vs. a repeatedly edited PNG. Every save of the JPG destroys information irretrievably, whereas every save of the PNG keeps it all. It's the same with MP3 vs. FLAC: MP3 throws away information each export, but FLAC doesn't. For an example, music artists/producers can't afford to lose information when they "bounce" a track. FLAC solves that. "People" don't need lossless, but for those that do, it's great.
- pwiecz 4y ago
  "People" don't need just lossless, they need The Beatles to release all of their pre-master tracks to be released in lossless format, so they of them can create their own remastered versions of a the tracks.
- michaelwww 4y ago
  16 bit is not enough?
  - deleted 4y ago
    [deleted]
  - pavon 4y ago
    No, if you want your final output to be 16-bit (eg CD quality), then you need more bits to work with when mixing. Each operation you perform will be requantized (rounded) to 16-bit resolution adding a bit of noise each time, until eventually the lower 4 bits (for example) are nothing but noise, and your signal is effectively only 12 bits. One the other hand if you did that mixing at 24-bit resolution, then you still have 20 bits of good data remaining, you throw away the bottom 4, and have nice clean 16-bit data to distribute. 32-bit integer is overkill in most situations, which is why it wasn't implemented for so long. 32-bit float is used a lot because it has 24-bits of resolution, but also has a huge scaling range, so you don't have to worry about clipping during intermediate operations, just at the end.
- squeaky-clean 4y ago
adgjlsfhk1 4y ago
There are definitely scientific uses that might want to (ab)use FLAC to store sound-like data that would benefit from 32 bit.
mmastrac 4y ago
There's a real use case for storing original tracks and masters in 32-bit and down-sampling for consumer use.
tombert 4y ago
While I doubt that human ears need 4 billion discrete levels, I think it's important for archival purposes.
- klodolph 4y ago
  Is it? What is your reasoning? And why would you pick 32-bit, instead of 24-bit or 64-bit or something else? Actual audio equipment has a noise floor. The encoding depth you use also has a noise floor. If the noise floor of the encoding is far below the noise floor of the signal, then it won't be perceptible. So you choose to put your noise floor at some amount below the existing noise floor. Not infinitely below, because that would require infinite bits. This is the reasoning you'd use for 24-bit audio, which has a very comfortable noise floor of -144 dB, which leaves a large margin even for extreme low-noise professional equipment (you might see -120 dB ish for extremely good equipment). At 24-bit, even passive components like transformers and resistors are contributing measurable amounts of noise. The reason you might pick 32-bit in practice is so that you can have lots of headroom for some DSP algorithm, or do lots of sums of different signals without accumulating quantization error. The final "archive" file will still have worse than 24-bit precision.
  - tombert 4y ago
    Fair enough! That's what I get for commenting on something I'm not as familiar enough with as I should be.
matheusmoreira 4y ago
How do you know no one needs it for anything? How do you know no one will ever need it in the future? It's not possible to know that. All we can do is capture the real world data with as much quality and precision as possible.
jlouis 4y ago
I don't think it's for storing music.
navjack27 4y ago
I got this one "remaster" of an album that I like that I've been working on that the 32-bit version that 100% definitely goes over 0 DB just happens to sound a whole lot better when I don't correct for that so on occasion I like to load up that album in Ableton live and listen to it exactly like that.

pavon 4y ago

Nice! I (ab)use FLAC to store waveform data that isn't audio, but compresses better using audio codecs than with general purpose compression algorithms. This will let me remove some hacks to compress 32-bit data as multiple channels.

colejohnson66 4y ago
What data are you compressing?
- nimish 4y ago
  I've used it to record oscilloscope captures. Especially useful since audio cards are cheap high res adc's.
  - mywittyname 4y ago
    I'm amazed that consumer-grade audio cards are still available and cheap, to be honest. I can't recall having seen one in years, then again, I haven't had a reason to look for them in years either.
    - wheels 4y ago
      To be clear: actual "PCI-slot"-ish audio cards are now rare. But USB / Thunderbolt (and previously Firewire) external interfaces are a dime a dozen. Serious audio mostly uses them now.
    - baybal2 4y ago
      Microchips are cheap, once initial investment is made. I guess, it only costs ~10¢ for new SoundBlasters at 90nm-130nm. The price of 10 sound cards will be enough to pay for a whole wafer.
    - zymhan 4y ago
      Creative is still marketing them to gamers, amazingly. Can't say I haven't been tempted.

Night_Thastus 4y ago

I had no idea FLAC didn't store 32-bit. I'll need to look over my library and make sure nothing strange is going on. I'm curious if sites like Qobuz will suddenly replace some of their existing files with higher-bit copies if they already had them and just had no way to encode them as FLAC.

klodolph 4y ago
It is extremely hard to justify 32-bit audio from a technical standpoint, since it requires that every component in your audio chain has better than 144 dB dynamic range. In practice, just about none of the analog components in your audio chain will have that kind of dynamic range, and any analog components in the recording chain are unlikely to have that range either. The kind of equipment I'd expect to see in order to record that kind of range is where you'd have something like multiple microphones, and switch back and forth between them based on the level of the material you're recording (which would have to be something like actual gunfire or explosions). I don't know what kind of equipment you'd need to reproduce this dynamic range, but I don't think it's audio equipment. My guess is that the main real application of this is going to be compression of non-audio data that happens to compress well with FLAC.
- simcop2387 4y ago
  I could see it as part of a pipeline/workflow where having that dynamic range lets you not worry about losing meaningful information through everything, even if you're going to compress it and take it back to 16bit cd quality at the end. Being able to losslessly store the results of each step in a FLAC file should be better than raw data since it'll be compressed and easier to manage then. That said, at the edges of that I totally agree even 24-bit can be questionable there since a lot of the analog side doesn't have a noise floor that would let it be meaningfully used.
  - klodolph 4y ago
    > That said, at the edges of that I totally agree even 24-bit can be questionable there since a lot of the analog side doesn't have a noise floor that would let it be meaningfully used. It's not that hard to beat 16-bit, which is 96 dB, using easy-to-find, off-the-shelf equipment. One example scenario is that you are recording something but you don't have a precise idea of how loud it will be ahead of time, so you record at low levels and rely on 24-bit capture to give you headroom above and noise floor below. Trying to capture at 16-bit can, in practice, be annoying and difficult because it is more likely that you will ruin takes by setting the gain wrong.

sprash 4y ago

Sadly still no support for variable blocksize.

nayuki 4y ago
I made my own encoder and experimented with variable block size (with dynamic programming) plus a massive amount of brute-force search for LPC parameters. But the amount of compression gain I could achieve over libflac was small and not worth the hours of encoding time. https://www.nayuki.io/page/benchmark-of-nayukis-flac-encoder https://www.nayuki.io/page/benchmark-of-nayukis-flac-encoder
- sprash 4y ago
  Sometimes encoding time doesn't matter. Archiving is done once in a lifetime. I encode everything with "--lax -8Vepl32" which takes ages but since it runs as a low priority background task I don't care. Also, you can use heuristics to choose the right blocksize instead of brute-forcing it. If the algorithm is clever enough you might achieve slightly better results at almost zero costs.
  - freedom-fries 4y ago
    Interesting. Do you have numbers to show that variable block size can be done with "almost zero cost" as you said?
    - nayuki 4y ago
      I think "If the algorithm is clever enough" is a big weasel phrase just like https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler . Having studied and written many algorithms, I have to conservatively assume that a clever algorithm doesn't exist unless it is proven by construction.

LeoPanthera 4y ago

32-bit integer, not float. I didn't even know 32-bit integer was a thing. Float is far more common. (For audio.)

survirtual 4y ago
This does not matter… 32bits is 32bits is 32bits.
- pavon 4y ago
  You are not going to get good compression reinterpreting a 32-bit float as if the bits were an integer.
  - survirtual 4y ago
    What? Most systems are 64bit now, I wouldn’t be doing signal processing on a system that wasn’t. If my signal has 32bits to work with during capture and my scratch space is 64bits during processing, I can assure you I am not losing any signal during conversion between int / float. What am I missing?
- LeoPanthera 4y ago
  It absolutely does matter when the compression algorithm is expecting audio in a certain format. If you put something else in those bits you will get worse, or no, compression, and might as well just use zip.
  - survirtual 4y ago
    I must be missing something here, but last I checked 32bit float uses about 8bits for metadata —- positive / negative, exponent, etc. — leaving 24bits for signal storage. A 32bit uint can just store the bits of raw data and would allow for denser packing of bits. You can then do processing with a 64bit float after converting the int’s bits without loss. Pretty sure every ADC IC I looked at uses integers. During analog processing, using the term “float” is highly disingenuous for an analog waveform, and if an ADC spouts out an int, what float are we talking about here? I maybe have gone too deep in the weeds reading TI chip manuals and writing drivers for some adc chips for my 8 channel, 32bit, 192kHz recording system, and maybe I missed some nuances in what “32bit” means, but last I checked my audio is pretty solid even after converting the 32bit int to 64bit float during my signal processing. Audio engineering has been a massive pain because these people use different terms for basic things (great example is “phantom power” — it’s just a 48v, low-current power supply, but good luck finding a simple answer to that useless term) and there doesn’t seem to be any authoritative references to find answers, so I am sure I am missing something here.

thrdbndndn 4y ago

I personally can't care less about 32-bit, but unfortunately a few digital music stores I buy from use this format (delivered in WAV). And I don't want to downscale them simply for archival reasons. Now I can at least re-compress them losslessly in FLAC. (previously I have to use Wavpack, which isn't bad but I'd prefer to have a single format in my media library.)

aaaaaaaaaaab 4y ago
Boomkat?
- thrdbndndn 4y ago
  I've heard of that too, but in my case it's batch of Japanese distributors like e-onkyo [1] or Ototoy [2]. [1] https://www.e-onkyo.com/music/ https://www.e-onkyo.com/music/ [2] https://ototoy.jp/top/ https://ototoy.jp/top/
  - astrange 4y ago
    Yes, unfortunately all of Japan has jumped headfirst into something called "hi-res audio" which is a complete scam, as CD audio is already the highest quality something mixed for humans could possibly need to be. I'd like to think it's an excuse to stop doing loudness war mastering, but it's probably just as bad. The real problem with CD audio is that it's premixed in stereo, and the real next step is Atmos/spatial audio. Just because you only have two ears doesn't mean you can't hear 360 degrees.
    - Maursault 4y ago
      > which is a complete scam I can hear the difference between 16-bit audio and 24-bit audio. 16-bit isn't bad at all, but 24-bit is definitely superior. I've worked with 32-bit on digital boards, and I think it is probably over-overkill, but whatever makes someone happy. But one thing I know for certain and beyond all possibility of doubt is that I only have two ears. All that is necessary for 3D sound is stereo speakers. To be clear, with only 2 speakers sound can be mixed to seem like it is coming from any direction the engineer desires, left, right, top, bottom, front, back, and everywhere in between. And stereo speakers are dead simple to set up correctly for 3D sound. Sure, you can use 6 or 8 or 10 speakers to get the same effect, if you can figure out how to properly set them up, which is not a given, but I think it's a dick-pull and a money grab because no one can hear with more than two ears. I realize there are Surround fanatics out there and in here that feel very strongly about it, but the physics of acoustics tells me their set ups are no better than and less efficient than stereo, and nearly always incorrectly configured.

aaaaaaaaaaab 4y ago

32bit float would be more interesting for music production.

ZoomZoomZoom 4y ago

Finally! I can ditch away 32b special casing in my project cleanup and backup automation! As much as I love WavPack[1] (it's great and has 32 bit integer/fp support since forever), Flac is much faster and probably more future-proof. Thanks a lot everyone at XIPH, your work really makes a difference. Edit: oops, I can't. No fp support yet. 1. https://www.wavpack.com/ https://www.wavpack.com/

ksec 4y ago
It is sad that a technically superior and patent free technology often lose out in the popularity contest. Sigh.
- dTal 4y ago
  What do you mean? Both FLAC and WavPack claim to be patent-free.
  - ksec 4y ago
    I am not aware of FLAC claims to be patent free ( when it started ), it may be royalty free and patent free now given it has been 20+ years since 1.0. They do however claims they dont hold any patent on it. WavePack was specifically designed to be Patent Free, or using old and expired Patent techniques.

procodr 4y ago

And iTunes still does not support FLAC. .