4 ms·

which math package are you using? Mathematica, Macsyma, ...? also, I have uBlock origin, and the imgur .png link shows me the page without the png... (EDIT: I

which math package are you using? Mathematica, Macsyma, ...? also, I have uBlock origin, and the imgur .png link shows me the page without the png... (EDIT: I just used wget on the exact same URL and it just downloads the png, it boggles my mind why firefox doesn't just show the PNG, or is this imgur detecting user agent etc?) Also, that's a good observation, that the spatial distribution of variance is noticable. Would you mind sharing the code for the last 2 plots you generated? For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Is it because you need to devise a new family of probability functions so that the expectation value in the optical domain is linearly proportional to the input optical value while having the bins spread according to the gamma curve?

nightcracker 7y ago
The plots in the linked StackExchange question were created using Wolfram Notebooks.
nightcracker 7y ago
> Would you mind sharing the code for the last 2 plots you generated? I just quickly edited my code to have OP's method, but didn't keep it after generating the plot. The code is generally a mess, because I kept experimenting with different methods, but this is the current version: https://www.shadertoy.com/view/wlX3RS https://www.shadertoy.com/view/wlX3RS > For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Because I believe it is only due to linear effects that 'dither + quantize' is a legal proper operation that does what you want. As soon as you enter a non-linear space this no longer holds, and you must see "dithered quantization" as a single operation, rather than adding independent noise followed by an oblivious quantization. So the issue is that our quantization must happen in sRGB space - our output space. Yet the probability distribution according to which we'd like to quantize follows rules formulated in a linear space, which requires a non-trivial and non-linear conversion. There is a very large family of probability distributions that have the appropriate mean (mean computed in linear space), but I believe if we also try to minimize the fluctuation in variance there should pop out a unique solution. In a fully linear world this is the triangular distribution.
- DoctorOetker 7y ago
  So if I understand correctly, its a mathematical question: What function family p(c,k) for integers k satisfies the following conditions: ExpectationValue < p(c,k) * e2o(k) > = c (for nearly all c) (over k) Variance < p(c,k) * k > = constant over nearly all c i.e. the expected variance should be constant in the perceptual (gamma compressed) domain, while the expected optical intensity should be the non-dithered original input optical intensity c. May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution.
  - nightcracker 7y ago
    My current probability distribution is just a bunch of curve fitting and solving the equations as I put them in the StackExchange question. I don't sum over all k, I only sum over k - 1, k and k + 1 where k is the closest integer to c. I don't believe you can get constant variance, due to the endpoints, where if you wish to have a correct mean (non-negotiable) the variance must approach 0. Therefore I believe the optimal variance curve to be constant in the middle with a small curve towards 0 at the endpoints. > May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution. The reason it's called this way is because in a linear space this effect can be achieved by a regular additive dither using a noise source that follows the [triangular distribution](https://en.wikipedia.org/wiki/Triangular_distribution https://en.wikipedia.org/wiki/Triangular_distribution). Why? I don't know I got from reading some audio dithering material. But plotting the results does give beautiful near-constant variance dithering, if we ignore sRGB and assume a linear space, except at the endpoints. This can be corrected, see this question: https://computergraphics.stackexchange.com/a/8777/10515 https://computergraphics.stackexchange.com/a/8777/10515. That's the origin of the plotting code as well. You can recover the correct linear p(c, k) by doing a convolution of the triangular distribution with a unit box: https://i.imgur.com/2AKVS4B.png https://i.imgur.com/2AKVS4B.png
    - raphlinus 7y ago
      What happens if you do the triangle distribution in sRGB space, derive the nonlinear system transfer function, then invert that (numerically)? The middle step is something like $t^{-1}(\sum_k p(c, k) t(k))$.

nightcracker 7y ago

The plots in the linked StackExchange question were created using Wolfram Notebooks.

nightcracker 7y ago

> Would you mind sharing the code for the last 2 plots you generated? I just quickly edited my code to have OP's method, but didn't keep it after generating the plot. The code is generally a mess, because I kept experimenting with different methods, but this is the current version: https://www.shadertoy.com/view/wlX3RS https://www.shadertoy.com/view/wlX3RS > For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Because I believe it is only due to linear effects that 'dither + quantize' is a legal proper operation that does what you want. As soon as you enter a non-linear space this no longer holds, and you must see "dithered quantization" as a single operation, rather than adding independent noise followed by an oblivious quantization. So the issue is that our quantization must happen in sRGB space - our output space. Yet the probability distribution according to which we'd like to quantize follows rules formulated in a linear space, which requires a non-trivial and non-linear conversion. There is a very large family of probability distributions that have the appropriate mean (mean computed in linear space), but I believe if we also try to minimize the fluctuation in variance there should pop out a unique solution. In a fully linear world this is the triangular distribution.

DoctorOetker 7y ago
So if I understand correctly, its a mathematical question: What function family p(c,k) for integers k satisfies the following conditions: ExpectationValue < p(c,k) * e2o(k) > = c (for nearly all c) (over k) Variance < p(c,k) * k > = constant over nearly all c i.e. the expected variance should be constant in the perceptual (gamma compressed) domain, while the expected optical intensity should be the non-dithered original input optical intensity c. May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution.
- nightcracker 7y ago
  My current probability distribution is just a bunch of curve fitting and solving the equations as I put them in the StackExchange question. I don't sum over all k, I only sum over k - 1, k and k + 1 where k is the closest integer to c. I don't believe you can get constant variance, due to the endpoints, where if you wish to have a correct mean (non-negotiable) the variance must approach 0. Therefore I believe the optimal variance curve to be constant in the middle with a small curve towards 0 at the endpoints. > May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution. The reason it's called this way is because in a linear space this effect can be achieved by a regular additive dither using a noise source that follows the [triangular distribution](https://en.wikipedia.org/wiki/Triangular_distribution https://en.wikipedia.org/wiki/Triangular_distribution). Why? I don't know I got from reading some audio dithering material. But plotting the results does give beautiful near-constant variance dithering, if we ignore sRGB and assume a linear space, except at the endpoints. This can be corrected, see this question: https://computergraphics.stackexchange.com/a/8777/10515 https://computergraphics.stackexchange.com/a/8777/10515. That's the origin of the plotting code as well. You can recover the correct linear p(c, k) by doing a convolution of the triangular distribution with a unit box: https://i.imgur.com/2AKVS4B.png https://i.imgur.com/2AKVS4B.png
  - raphlinus 7y ago
    What happens if you do the triangle distribution in sRGB space, derive the nonlinear system transfer function, then invert that (numerically)? The middle step is something like $t^{-1}(\sum_k p(c, k) t(k))$.