2 ms·
The 0.5 comes from debiasing the round-trip. If your conversion from high precision -> 8-bit is just multiplication by 256 and then truncation, then you’ve got
by klodolph 4mo ago
The 0.5 comes from debiasing the round-trip.
If your conversion from high precision -> 8-bit is just multiplication by 256 and then truncation, then you’ve got the mid-riser quantizer. The +0.5 comes from interpreting a value of 0 as bucket from 0-1, just like the value of 255 is the bucket from 255-256. It’s introduced in the conversion back from 8-bit to high precision.
- dpark 4mo agoYou’re still misinterpreting what I proposed. I didn’t propose a +0.5 debias at all. I proposed that the bias is removed by scaling. In this case you divide by 255.0 but multiply by 256. But again, the likely reason no one does this is because it introduces a bias in the other direction, toward 255.
- klodolph 4mo agoIt’s unbiased, but amplifies the signal.