4 ms·

This is not correct dithering, because uniform noise is not correct. You need triangular noise, and have it somehow be aware of sRGB. Why? Because otherwise yo

This is not correct dithering, because uniform noise is not correct. You need triangular noise, and have it somehow be aware of sRGB. Why? Because otherwise your variance is not uniform in your dithering spectrum, and gives noticeable artifacts. So it's not correct to only choose from floor(c) and floor(c)+1, you actually need to choose from round(c) - 1, round(c) and round(c) + 1 with the right probabilities. I've gotten stuck on this for days, and even tried to ask on stackexchange, but to no avail: https://math.stackexchange.com/questions/3200249/maintaining-probability-distribution-under-transform https://math.stackexchange.com/questions/3200249/maintaining... Yet this is important, as dithering with triangular noise gives vastly superior dithering. This is 5 grayscale values with my best found method (which is not fully mathematically correct, but a close approximation): https://i.imgur.com/sNq20Oo.png https://i.imgur.com/sNq20Oo.png The yellow line indicates sRGB signal, the red the error, and the blue the variance. Now compare it to OP's method: https://i.imgur.com/FjBUuCr.png https://i.imgur.com/FjBUuCr.png Notice the significant banding? It's because there are regions where the variance of the dither essentially hits zero, producing a (near) pure color band that's very noticeable.

DoctorOetker 7y ago
which math package are you using? Mathematica, Macsyma, ...? also, I have uBlock origin, and the imgur .png link shows me the page without the png... (EDIT: I just used wget on the exact same URL and it just downloads the png, it boggles my mind why firefox doesn't just show the PNG, or is this imgur detecting user agent etc?) Also, that's a good observation, that the spatial distribution of variance is noticable. Would you mind sharing the code for the last 2 plots you generated? For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Is it because you need to devise a new family of probability functions so that the expectation value in the optical domain is linearly proportional to the input optical value while having the bins spread according to the gamma curve?
- nightcracker 7y ago
  The plots in the linked StackExchange question were created using Wolfram Notebooks.
- nightcracker 7y ago
  > Would you mind sharing the code for the last 2 plots you generated? I just quickly edited my code to have OP's method, but didn't keep it after generating the plot. The code is generally a mess, because I kept experimenting with different methods, but this is the current version: https://www.shadertoy.com/view/wlX3RS https://www.shadertoy.com/view/wlX3RS > For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Because I believe it is only due to linear effects that 'dither + quantize' is a legal proper operation that does what you want. As soon as you enter a non-linear space this no longer holds, and you must see "dithered quantization" as a single operation, rather than adding independent noise followed by an oblivious quantization. So the issue is that our quantization must happen in sRGB space - our output space. Yet the probability distribution according to which we'd like to quantize follows rules formulated in a linear space, which requires a non-trivial and non-linear conversion. There is a very large family of probability distributions that have the appropriate mean (mean computed in linear space), but I believe if we also try to minimize the fluctuation in variance there should pop out a unique solution. In a fully linear world this is the triangular distribution.
  - DoctorOetker 7y ago
    So if I understand correctly, its a mathematical question: What function family p(c,k) for integers k satisfies the following conditions: ExpectationValue < p(c,k) * e2o(k) > = c (for nearly all c) (over k) Variance < p(c,k) * k > = constant over nearly all c i.e. the expected variance should be constant in the perceptual (gamma compressed) domain, while the expected optical intensity should be the non-dithered original input optical intensity c. May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution.

DoctorOetker 7y ago

which math package are you using? Mathematica, Macsyma, ...? also, I have uBlock origin, and the imgur .png link shows me the page without the png... (EDIT: I just used wget on the exact same URL and it just downloads the png, it boggles my mind why firefox doesn't just show the PNG, or is this imgur detecting user agent etc?) Also, that's a good observation, that the spatial distribution of variance is noticable. Would you mind sharing the code for the last 2 plots you generated? For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Is it because you need to devise a new family of probability functions so that the expectation value in the optical domain is linearly proportional to the input optical value while having the bins spread according to the gamma curve?

nightcracker 7y ago
The plots in the linked StackExchange question were created using Wolfram Notebooks.
nightcracker 7y ago
> Would you mind sharing the code for the last 2 plots you generated? I just quickly edited my code to have OP's method, but didn't keep it after generating the plot. The code is generally a mess, because I kept experimenting with different methods, but this is the current version: https://www.shadertoy.com/view/wlX3RS https://www.shadertoy.com/view/wlX3RS > For incorporating gamma, and using the original HN post terminology, what prevents you from generalizing your technique to the optical (linear) domain instead of the electronic (gamma) domain? Because I believe it is only due to linear effects that 'dither + quantize' is a legal proper operation that does what you want. As soon as you enter a non-linear space this no longer holds, and you must see "dithered quantization" as a single operation, rather than adding independent noise followed by an oblivious quantization. So the issue is that our quantization must happen in sRGB space - our output space. Yet the probability distribution according to which we'd like to quantize follows rules formulated in a linear space, which requires a non-trivial and non-linear conversion. There is a very large family of probability distributions that have the appropriate mean (mean computed in linear space), but I believe if we also try to minimize the fluctuation in variance there should pop out a unique solution. In a fully linear world this is the triangular distribution.
- DoctorOetker 7y ago
  So if I understand correctly, its a mathematical question: What function family p(c,k) for integers k satisfies the following conditions: ExpectationValue < p(c,k) * e2o(k) > = c (for nearly all c) (over k) Variance < p(c,k) * k > = constant over nearly all c i.e. the expected variance should be constant in the perceptual (gamma compressed) domain, while the expected optical intensity should be the non-dithered original input optical intensity c. May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution.