3 ms·

https://gist.github.com/deckar01/f77d98550eaf5d9b3a954eb034388f86?permalink_comment_id=5627238#gistcomment-5627238 https://gist.github.com/deckar01/f77d98550eaf

https://gist.github.com/deckar01/f77d98550eaf5d9b3a954eb034388f86?permalink_comment_id=5627238#gistcomment-5627238 https://gist.github.com/deckar01/f77d98550eaf5d9b3a954eb0343... Here is a visualization I made recently on the density of float32. It seems that float32 is basically just PCM, which was a lossy audio compression exploiting the fact that human hearing has logarithmic sensitivity. I’m not sure why they needed the mantissa though. If you give all 31 bits to the exponent, then normalize it to +/-2^7, you get a continuous version of the same function.

tialaramex 1y ago
So, PCM isn't the thing you meant here. PCM just means Pulse-code modulation, you're probably thinking of a specific non-linear PCM and maybe that was even the default for some particular hardware or software you used, but that's not what PCM itself means and these days almost everything uses Linear PCM.
- userbinator 1y ago
  I think G.711 PCM is what the OP meant.
  - tialaramex 1y ago
    Wow. G.711 is extremely obsolete. Interpreting PCM as G.711 (which is from the 1970s) is about similar to if somebody said "Windows" but meant Windows 2.x the 1980s DOS-based Microsoft GUI. I guess I don't have to feel like I'm the oldest person reading HN.
    - userbinator 1y ago
      In the telco industry, "PCM" or more precisely "PCMA" and "PCMU", refers to G.711. It's still the default fallback for VoIP applications.
timewizard 1y ago
> It seems that float32 is basically just PCM float has higher accuracy around 1.0 than around 2*24. This makes it quite a bit different from PCM which is fully linear. Which is probably why floating point PCM keeps it's samples primarily between -1.0 and +1.0. > which was a lossy audio compression It's not lossy. Your bit depth simply defines the noise floor which is the smallest difference in volume you can represent. This may result in loss of information but at even 16 bits only the most sensitive of ears could even pretend to notice. > If you give all 31 bits to the exponent, then normalize it to +/-2^7, you get a continuous version of the same function. You'll extend the range but loose all the precision. This is probably the opposite of what any IEE754 user actually wants.
- dist-epoch 1y ago
  > Which is probably why floating point PCM keeps it's samples primarily between -1.0 and +1.0. No, it's just it's more natural/intuitive to express algorithms in a normalized range if given the possibility. Same with floating point RGBA (like in GPUs)
- deckar01 1y ago
  Here is what a 31-bit exponent 0-bit mantissa encoding looks like compared to float32: https://gist.github.com/deckar01/3f93802329debe116b0c3570bed65de2?permalink_comment_id=5647878#gistcomment-5647878 https://gist.github.com/deckar01/3f93802329debe116b0c3570bed...
  - timewizard 1y ago
    I don't have the time to fully analyze this but my concern would be here: exponent *= 2 ** (8 - E) In the E=8 case then this is just `* 1`. In the E=31 case this is now `* 2*-23`. Python is going to do all of this for you in the float64 domain. I think it's possible that you haven't graphed what you intended. You also don't have subnormals, infinities or propagating NaNs. You manage to only retain the signed 0. EDIT: And the midpoint of your system is 0.5. Which is a little uncomfortable.
GrantMoyer 1y ago
Without a mantissa, way too much precision is allocated to the near zero range and not enough to the "near infinity" range. Consider that without a mantissa, the second largest float is only half of the largest float. With a 23 bit mantissa, there are 2^23 floats from half the largest to the largest.
- deckar01 1y ago
  You could change the scaling factor to target any bounds you want. On average the precision is equal. The mantissa just adds linear segments to a logarithmic curve.
  - GrantMoyer 1y ago
    > On average the precision is equal. The mantissa just adds linear segments to a logarithmic curve. Yes, exactly; the linear regions are needed to more evenly distribute precision, while the average precision remains the same. Alternatively, you can omit the mantissa, but use an exponent base much closer to 1 (perhaps 1 + 2⁻²³).

tialaramex 1y ago

So, PCM isn't the thing you meant here. PCM just means Pulse-code modulation, you're probably thinking of a specific non-linear PCM and maybe that was even the default for some particular hardware or software you used, but that's not what PCM itself means and these days almost everything uses Linear PCM.

userbinator 1y ago
I think G.711 PCM is what the OP meant.
- tialaramex 1y ago
  Wow. G.711 is extremely obsolete. Interpreting PCM as G.711 (which is from the 1970s) is about similar to if somebody said "Windows" but meant Windows 2.x the 1980s DOS-based Microsoft GUI. I guess I don't have to feel like I'm the oldest person reading HN.
  - userbinator 1y ago
    In the telco industry, "PCM" or more precisely "PCMA" and "PCMU", refers to G.711. It's still the default fallback for VoIP applications.

timewizard 1y ago

> It seems that float32 is basically just PCM float has higher accuracy around 1.0 than around 2*24. This makes it quite a bit different from PCM which is fully linear. Which is probably why floating point PCM keeps it's samples primarily between -1.0 and +1.0. > which was a lossy audio compression It's not lossy. Your bit depth simply defines the noise floor which is the smallest difference in volume you can represent. This may result in loss of information but at even 16 bits only the most sensitive of ears could even pretend to notice. > If you give all 31 bits to the exponent, then normalize it to +/-2^7, you get a continuous version of the same function. You'll extend the range but loose all the precision. This is probably the opposite of what any IEE754 user actually wants.

dist-epoch 1y ago
> Which is probably why floating point PCM keeps it's samples primarily between -1.0 and +1.0. No, it's just it's more natural/intuitive to express algorithms in a normalized range if given the possibility. Same with floating point RGBA (like in GPUs)
deckar01 1y ago
Here is what a 31-bit exponent 0-bit mantissa encoding looks like compared to float32: https://gist.github.com/deckar01/3f93802329debe116b0c3570bed65de2?permalink_comment_id=5647878#gistcomment-5647878 https://gist.github.com/deckar01/3f93802329debe116b0c3570bed...
- timewizard 1y ago
  I don't have the time to fully analyze this but my concern would be here: exponent *= 2 ** (8 - E) In the E=8 case then this is just `* 1`. In the E=31 case this is now `* 2*-23`. Python is going to do all of this for you in the float64 domain. I think it's possible that you haven't graphed what you intended. You also don't have subnormals, infinities or propagating NaNs. You manage to only retain the signed 0. EDIT: And the midpoint of your system is 0.5. Which is a little uncomfortable.

GrantMoyer 1y ago

Without a mantissa, way too much precision is allocated to the near zero range and not enough to the "near infinity" range. Consider that without a mantissa, the second largest float is only half of the largest float. With a 23 bit mantissa, there are 2^23 floats from half the largest to the largest.

deckar01 1y ago
You could change the scaling factor to target any bounds you want. On average the precision is equal. The mantissa just adds linear segments to a logarithmic curve.
- GrantMoyer 1y ago
  > On average the precision is equal. The mantissa just adds linear segments to a logarithmic curve. Yes, exactly; the linear regions are needed to more evenly distribute precision, while the average precision remains the same. Alternatively, you can omit the mantissa, but use an exponent base much closer to 1 (perhaps 1 + 2⁻²³).