3 ms·

> in every shot that panned, I couldn’t stop fixating on the incorrect twinkling of all the stars I wish he had included an example screenshot or video clip of

> in every shot that panned, I couldn’t stop fixating on the incorrect twinkling of all the stars I wish he had included an example screenshot or video clip of what’s wrong with them.

d_tr 3y ago
Two adjacent gray pixels (the star is between two pixels) won't have the same total brightness as a single fully bright one (the star is at the center of a single pixel) if the interpolation is not correct with respect to gamma. So, as the star moves slowly, I would expect some twinkling. This is probably the issue he is referring to.
- NotYourLawyer 3y ago
  That makes sense, I bet you’re right.
- robinsord 3y ago
  Isn't this entirely dependent on the sampling method used to upscale...
meindnoch 3y ago
Consider moving a 1 lumen Dirac impulse across the screen. It looks like this when it moves perfectly with 1 pixel per frame: frame=0: 1 0 0 0 frame=1: 0 1 0 0 frame=2: 0 0 1 0 frame=3: 0 0 0 1 Values are in lumens. Now let's do it half as fast: frame=0: 1 0 0 0 frame=1: 0.5 0.5 0 0 frame=2: 0 1 0 0 frame=3: 0 0.5 0.5 0 frame=4: 0 0 1 0 frame=5: 0 0 0.5 0.5 frame=6: 0 0 0 1 When the impulse falls between two pixels, the amount of light should be distributed over those two pixels, so that the total amount emitted stays the same. However, we use a nonlinear encoding to map these physical values to numbers, which is called gamma compression. So in case of an 8-bit encoding, if 1 lumen is mapped to 255, then 0.5 lumen would correspond to a value of say 186, instead of 128. The display hardware uses the reverse of this mapping to emit the intended amount of light from each display pixel, so everything works fine as long as gamma is accounted for in the whole pipeline. However, if you have a gamma-compressed image, you must be careful when doing any transformations on this image in the digital domain, because the pixel values live in a non-linear space, and this must be taken into account when transforming them! E.g. let's say we have our 0.5 pixel/frame animation from above, encoded with gamma=0.45: frame=0: 255 0 0 0 frame=1: 186 186 0 0 frame=2: 0 255 0 0 frame=3: 0 186 186 0 frame=4: 0 0 255 0 frame=5: 0 0 186 186 frame=6: 0 0 0 255 And in post-processing you resize this video to 0.5x of the original size, with software that naïvely blends the pixel values when shrinking the image: frame=0: 128 0 frame=1: 186 0 frame=2: 128 0 frame=3: 93 93 frame=4: 0 128 frame=5: 0 186 frame=6: 0 128 Oops! The intensity is no longer constant, which is going to be visible as twinkling when viewing the video.
ReactiveJelly 3y ago
(The sibling comments are good but I want to add this anyway) Wikipedia has a static test image for gamma here https://en.wikipedia.org/wiki/File:Gamma_correction_test_picture.png https://en.wikipedia.org/wiki/File:Gamma_correction_test_pic... If you look in the green section, you can see the background is a checkerboard of (0, 0, 0) black and (0, 255, 0) green. Because of gamma, this is not equivalent to (0, 127, 0). The 2.6, 2.2, 1.8 sections are supposed to blend with the background if that's the gamma that your image-to-screen pipeline uses. (Including the browser's PNG decoder, the browser's painting and compositing, the desktop's painting and compositing, anything goofy the GPU drivers do, and all the goofy stuff monitors do) Looks like my cheapo 2010 LCD's gamma changes greatly depending on viewing angle. Ugh :S
- panzi 3y ago
  Funny thing: When I view that image and zoom in on my 4k monitor it is somehow interpolated. I have a bookmarklet that sets the style `image-rendering: pixelated` and when I click that it suddenly gets much brighter. That interpolation doesn't just look ugly blurry, it corrupts the colors. Maybe instead of using a bookmarklet I should put that into a user style applied to all websites. The bookmarklet for anyone who is interested: javascript:void(document.body.style.imageRendering='pixelated')

> in every shot that panned, I couldn’t stop fixating on the incorrect twinkling of all the stars I wish he had included an example screenshot or video clip of what’s wrong with them.

d_tr 3y ago

Two adjacent gray pixels (the star is between two pixels) won't have the same total brightness as a single fully bright one (the star is at the center of a single pixel) if the interpolation is not correct with respect to gamma. So, as the star moves slowly, I would expect some twinkling. This is probably the issue he is referring to.

NotYourLawyer 3y ago
That makes sense, I bet you’re right.
robinsord 3y ago
Isn't this entirely dependent on the sampling method used to upscale...

meindnoch 3y ago

Consider moving a 1 lumen Dirac impulse across the screen. It looks like this when it moves perfectly with 1 pixel per frame: frame=0: 1 0 0 0 frame=1: 0 1 0 0 frame=2: 0 0 1 0 frame=3: 0 0 0 1 Values are in lumens. Now let's do it half as fast: frame=0: 1 0 0 0 frame=1: 0.5 0.5 0 0 frame=2: 0 1 0 0 frame=3: 0 0.5 0.5 0 frame=4: 0 0 1 0 frame=5: 0 0 0.5 0.5 frame=6: 0 0 0 1 When the impulse falls between two pixels, the amount of light should be distributed over those two pixels, so that the total amount emitted stays the same. However, we use a nonlinear encoding to map these physical values to numbers, which is called gamma compression. So in case of an 8-bit encoding, if 1 lumen is mapped to 255, then 0.5 lumen would correspond to a value of say 186, instead of 128. The display hardware uses the reverse of this mapping to emit the intended amount of light from each display pixel, so everything works fine as long as gamma is accounted for in the whole pipeline. However, if you have a gamma-compressed image, you must be careful when doing any transformations on this image in the digital domain, because the pixel values live in a non-linear space, and this must be taken into account when transforming them! E.g. let's say we have our 0.5 pixel/frame animation from above, encoded with gamma=0.45: frame=0: 255 0 0 0 frame=1: 186 186 0 0 frame=2: 0 255 0 0 frame=3: 0 186 186 0 frame=4: 0 0 255 0 frame=5: 0 0 186 186 frame=6: 0 0 0 255 And in post-processing you resize this video to 0.5x of the original size, with software that naïvely blends the pixel values when shrinking the image: frame=0: 128 0 frame=1: 186 0 frame=2: 128 0 frame=3: 93 93 frame=4: 0 128 frame=5: 0 186 frame=6: 0 128 Oops! The intensity is no longer constant, which is going to be visible as twinkling when viewing the video.

ReactiveJelly 3y ago

(The sibling comments are good but I want to add this anyway) Wikipedia has a static test image for gamma here https://en.wikipedia.org/wiki/File:Gamma_correction_test_picture.png https://en.wikipedia.org/wiki/File:Gamma_correction_test_pic... If you look in the green section, you can see the background is a checkerboard of (0, 0, 0) black and (0, 255, 0) green. Because of gamma, this is not equivalent to (0, 127, 0). The 2.6, 2.2, 1.8 sections are supposed to blend with the background if that's the gamma that your image-to-screen pipeline uses. (Including the browser's PNG decoder, the browser's painting and compositing, the desktop's painting and compositing, anything goofy the GPU drivers do, and all the goofy stuff monitors do) Looks like my cheapo 2010 LCD's gamma changes greatly depending on viewing angle. Ugh :S

panzi 3y ago
Funny thing: When I view that image and zoom in on my 4k monitor it is somehow interpolated. I have a bookmarklet that sets the style `image-rendering: pixelated` and when I click that it suddenly gets much brighter. That interpolation doesn't just look ugly blurry, it corrupts the colors. Maybe instead of using a bookmarklet I should put that into a user style applied to all websites. The bookmarklet for anyone who is interested: javascript:void(document.body.style.imageRendering='pixelated')