4 ms·

My current probability distribution is just a bunch of curve fitting and solving the equations as I put them in the StackExchange question. I don't sum over al

My current probability distribution is just a bunch of curve fitting and solving the equations as I put them in the StackExchange question. I don't sum over all k, I only sum over k - 1, k and k + 1 where k is the closest integer to c. I don't believe you can get constant variance, due to the endpoints, where if you wish to have a correct mean (non-negotiable) the variance must approach 0. Therefore I believe the optimal variance curve to be constant in the middle with a small curve towards 0 at the endpoints. > May I ask how you came up with the current probability distribution? you call it triangular but it's not really a hat distribution. The reason it's called this way is because in a linear space this effect can be achieved by a regular additive dither using a noise source that follows the [triangular distribution](https://en.wikipedia.org/wiki/Triangular_distribution https://en.wikipedia.org/wiki/Triangular_distribution). Why? I don't know I got from reading some audio dithering material. But plotting the results does give beautiful near-constant variance dithering, if we ignore sRGB and assume a linear space, except at the endpoints. This can be corrected, see this question: https://computergraphics.stackexchange.com/a/8777/10515 https://computergraphics.stackexchange.com/a/8777/10515. That's the origin of the plotting code as well. You can recover the correct linear p(c, k) by doing a convolution of the triangular distribution with a unit box: https://i.imgur.com/2AKVS4B.png https://i.imgur.com/2AKVS4B.png

raphlinus 7y ago
What happens if you do the triangle distribution in sRGB space, derive the nonlinear system transfer function, then invert that (numerically)? The middle step is something like $t^{-1}(\sum_k p(c, k) t(k))$.
DoctorOetker 7y ago
I've worked it out on paper, both for functionally known gamma functions (optionally piecewise defined) and for arbitrary monotonically increasing gamma arrays (where naive linear interpolation is used for values of c that do not correspond to integral k's). I will try to write it out in LaTeX, and will link when finished. Sadly the shadertoy site doesn't work on my smarthone (it complains that uint's are not available before mobile GLSL 3.0, I tried modifying them all to normal int's, but then got stuck on the hash function through which I assume you generate pseudo random numbers.
- nightcracker 7y ago
  Did you end up writing it down?

raphlinus 7y ago

What happens if you do the triangle distribution in sRGB space, derive the nonlinear system transfer function, then invert that (numerically)? The middle step is something like $t^{-1}(\sum_k p(c, k) t(k))$.

DoctorOetker 7y ago

I've worked it out on paper, both for functionally known gamma functions (optionally piecewise defined) and for arbitrary monotonically increasing gamma arrays (where naive linear interpolation is used for values of c that do not correspond to integral k's). I will try to write it out in LaTeX, and will link when finished. Sadly the shadertoy site doesn't work on my smarthone (it complains that uint's are not available before mobile GLSL 3.0, I tried modifying them all to normal int's, but then got stuck on the hash function through which I assume you generate pseudo random numbers.

nightcracker 7y ago
Did you end up writing it down?