6 ms·

The tradeoff of rendering or filming motion blur at finite refresh rates is the audience can move their eyes to follow an object moving around the screen. In re

by geor9e 3y ago

The tradeoff of rendering or filming motion blur at finite refresh rates is the audience can move their eyes to follow an object moving around the screen. In real life, that causes the object to become sharp. So, you either need to track eye motion and blur according to relative motion, or do no motion blur at an infinite refresh rate. Neither is practical with todays's technology, so it's always going to look wrong. A good director or game designer will choose shutter rate or render blur according to how they expect the audience eyes to be moving.

chaboud 3y ago
However, the audience is used to shutter angle as part of the visual vernacular (e.g., narrow shutter angle for hyper edgy rap videos, long shutter angle for dreamy retro vibes). If rendered content can't speak the same visual language, a tool is missing, regardless of framerate (up to a point... At 400Hz, I'd be impressed by someone really seeing the difference). What's interesting about rendered content is that it can extend that (e.g., a shutter angle beyond the duration of a frame), playing with something we thought we had a handle on.
- drc500free 3y ago
  I first learned about shutter angle when reading about its use in Saving Private Ryan; the narrow 45 and 90 degree shutters made the opening scenes much more visceral with so much flying through the air. https://cinemashock.org/2012/07/30/45-degree-shutter-in-saving-private-ryan/ https://cinemashock.org/2012/07/30/45-degree-shutter-in-savi...
  - arrowsmith 3y ago
    That was a really interesting article, thanks!
- CarVac 3y ago
  It's also used for subject isolation in much the same way as depth of field. Only objects not moving relative to the frame can be seen sharply.
- orlp 3y ago
  > At 400Hz, I'd be impressed by someone really seeing the difference. Trivial. Drag your white cursor quickly across the screen against a black background. You will see clear gaps between the individual cursor afterimages on your retina. Double the FPS, halve the size of the gaps. On a 240Hz monitor I can so clearly see gaps such that if they were halved the gaps would still easily be visible. Ergo 400Hz would still easily be distinguishable from continuous motion. To put numbers on this, consider a vertical line 1 pixel wide moving across a 4K screen in 1 second (that's not even that fast). At 480Hz that's a shift of 8 pixels per frame. So you'd need at least 8x the framerate for motion of this line to be continuous.
  - zozbot234 3y ago
    > Trivial. Drag your white cursor quickly across the screen against a black background. You will see clear gaps between the individual cursor afterimages on your retina. That's the outcome of aliasing, not of the FPS limitation itself. You could analytically supersample the motion in the time domain and then blur it just enough to remove the aliasing, and the distinct images would then disappear. Motion blur approximates the same result.
    - blauditore 3y ago
      That's not true at all. When moving multiple pixels per frame, there will always be visible jumps. Aliasing only becomes relevant at the pixel level.
      - zozbot234 3y ago
        "Pixels" and "frames" are the exact same thing analytically, only in the spatial vs. time domain. This is very much an instance of aliasing, which is why blur happens to correct it.
        blauditore 3y ago
        Alright, but supersampling would exactly achieve another approximation to (real) motion blur, but still suffer from the same issues (like not becoming sharp when tracking with your eyes).
- harimau777 3y ago
  Can you recommend any resources on how shutter angle is used to communicate different messages in film? The Wikipedia article only gives some very basic examples (short shutter angle when you want to capture particles in the air or for action scenes).
  - chaboud 3y ago
    https://www.indepthcine.com/videos/shutter-angle https://www.indepthcine.com/videos/shutter-angle
pierrec 3y ago
Exactly, while doing background research for this I read a really neat paper that describes what you're saying, and offers a solution based on trying to predict the viewer's eye motion: Temporal Video Filtering and Exposure Control for Perceptual Motion Blur (Stengel et al., 2015)
Veedrac 3y ago
You can't properly make a moving image sharp at a finite refresh rate (though you can approach it for sufficiently high ones), because the object can only move in a temporally juddered path, which doesn't match the eye's movement.
- planede 3y ago
  You can with a strobed backlight.
  - Veedrac 3y ago
    Fair point.
nyanpasu64 3y ago
Moving objects can become even sharper if each frame is displayed in a fixed location for a shorter period of time (reduced MPRT), preventing eye-tracking motion from smearing each displayed frame. This can be achieved through CRT/OLED scanout (often rolling) or LCD backlight strobing (usually full-frame by necessity). Unfortunately displaying each frame for a short time is unbearably flickery at 24 Hz (so movie projectors would show film frames 2-3 times), just barely tolerable at 50 (to the point some European CRT televisions frame-doubled 50 Hz video to 100 Hz, causing objects to appear doubled in motion), and ideally needs 70-75 Hz or above for optimally fluid motion and minimum eyestrain (which can't show 60 FPS recorded video without judder and/or tearing).
- s4i 3y ago
  > to the point some European CRT televisions frame-doubled 50 Hz video to 100 Hz, causing objects to appear doubled in motion Hmm, this is not accurate (or I don't understand what you mean). 100Hz CRT TVs available in the 90s/00s did not interpolate frames to get smoother motion – they only existed to reduce flicker. I think such TVs also existed in NTSC markets (120Hz)? Anyway, ever since the late 00s, pretty much all the TVs you can buy from a store do come with an interpolation algorithm to artificially display a higher frame rate image (e.g. 100Hz/120Hz) from a lower frame rate source (e.g. 23.976/24/29.97/30/50/59.94/60 fps) – which (personal opinion) looks terrible (and can be turned off from the settings – but the default is always on). This is an interesting side tangent when it comes to motion blur, because the blur is prebaked in the input signal and cannot be easily removed. Thus, the end result always has an artificial look. For instance, if the source material is shot in 24fps with the typical 180 degree shutter angle, each frame spans 41.6ms of which the shutter was open for 20.8ms. Then your TV interpolates that to be 96Hz or whatever. However, the individual output frames still look (or, with the added artifacts etc., mostly look) like the shutter was open for 20.8ms per frame. However, each frame now spans 10.2ms which is a shorter time than the shutter speed!
  - cubefox 3y ago
    As far as I understand, you get doubling not because of any interpolation but because you track objects with your eye. That means the tracked object is in rest in your vision. Which means the screen actually moves relative to your vision. Which means the object smears, where the pixel length of the smear is equal to the pixel distance the tracked object moves in the time the frame is visible on screen. On CRT there is less smear, because the frames are only flashed for a short time on the otherwise dark screen. But if the FPS are not equal to the display Hz, e.g. a half, the tracked object appears on multiple, e.g. two, locations on the screen. Because you show the frame twice, and if you track an object, the screen has moved (relative to your vision) for a few pixels by the time you show the second frame. This happens e.g. if you play a 30 FPS game on a 60 Hz CRT, or when you watch 50 FPS content on a 100 Hz screen. Though the effect isn't usually as dramatic as it may sound here. Doubling is also not as bad as smearing, e.g. in the case of scrolling text.
cubefox 3y ago
An alternative to infinite refresh rate is to show each frame only a fraction of the normal frame time, i.e. to quickly "flash" the individual frames and showing a black screen otherwise. This reduces the maximum screen brightness, an it requires a minimum frame rate to avoid flicker, but it reduces the type of "eye tracking blur" (persistence blur) which isn't present in real life. To be precise, to completely remove the tracking blur you would need to flash each frame only for an infinitesimal time period, which of course isn't realistic. But VR headsets do use this flashing/strobing approach. By the way, this is also the reason why CRT and Plasma screens had much better motion clarity than LCD or OLED. The former flash each frame for a short time, while the latter "sample and hold" the frame for the entire frame time (e.g. 1/60th of a second for 60 Hz). 60 FPS on a CRT looks probably more fluid than 120 FPS on an OLED. Another option for games is to indeed add a lot of frames by using reprojection techniques. This can approximate the real camera movements without needing to render a ton of expensive frames in the engine. This also is already used in VR, just currently not at overly high frame rates. This great article goes into more detail: https://blurbusters.com/frame-generation-essentials-interpolation-extrapolation-and-reprojection/ https://blurbusters.com/frame-generation-essentials-interpol... Something like 1000 FPS with reprojection are apparently quite realistic, which should solve the problem of tracking blur without reducing screen brightness.
- account42 3y ago
  > Something like 1000 FPS with reprojection are apparently quite realistic, which should solve the problem of tracking blur without reducing screen brightness. Is a 1000 FPS screen more realistic than a screen capable of the higher maximum brightness needed to compensate for black frame insertion though? HDR screens are already a thing and you could already gain persistence improvements there for LDR scenes without needing any new hardware by always driving pixels at max brighness but only for a reduced time depending on the target luminance. Or just reduce the ambient light enough - I even run even my LDR monitor at 10% brightness.
  - cubefox 3y ago
    As far as I understand, the only thing that matters for reducing unrealistic "tracking blur" is indeed how short a frame is displayed on screen. Which could be achieved by strobing / black frame insertion or by increasing frame rate. Or even both, as in some VR applications. So the effect should be the same, except for the brightness thing, which may not be a problem if there is a lot of screen brightness headroom anyway. For HDR LCD TVs the LED backlights get quite bright actually. OLED not so much. One advantage of higher frame rates (as opposed to strobing) would be quicker input responses to, e.g., moving the camera. That's not overly important on a normal screen but quite significant on a VR headset where we expect head movements to be represented very quickly.

by geor9e 3y ago

chaboud 3y ago

However, the audience is used to shutter angle as part of the visual vernacular (e.g., narrow shutter angle for hyper edgy rap videos, long shutter angle for dreamy retro vibes). If rendered content can't speak the same visual language, a tool is missing, regardless of framerate (up to a point... At 400Hz, I'd be impressed by someone really seeing the difference). What's interesting about rendered content is that it can extend that (e.g., a shutter angle beyond the duration of a frame), playing with something we thought we had a handle on.

drc500free 3y ago
I first learned about shutter angle when reading about its use in Saving Private Ryan; the narrow 45 and 90 degree shutters made the opening scenes much more visceral with so much flying through the air. https://cinemashock.org/2012/07/30/45-degree-shutter-in-saving-private-ryan/ https://cinemashock.org/2012/07/30/45-degree-shutter-in-savi...
- arrowsmith 3y ago
  That was a really interesting article, thanks!
CarVac 3y ago
It's also used for subject isolation in much the same way as depth of field. Only objects not moving relative to the frame can be seen sharply.
orlp 3y ago
> At 400Hz, I'd be impressed by someone really seeing the difference. Trivial. Drag your white cursor quickly across the screen against a black background. You will see clear gaps between the individual cursor afterimages on your retina. Double the FPS, halve the size of the gaps. On a 240Hz monitor I can so clearly see gaps such that if they were halved the gaps would still easily be visible. Ergo 400Hz would still easily be distinguishable from continuous motion. To put numbers on this, consider a vertical line 1 pixel wide moving across a 4K screen in 1 second (that's not even that fast). At 480Hz that's a shift of 8 pixels per frame. So you'd need at least 8x the framerate for motion of this line to be continuous.
- zozbot234 3y ago
  > Trivial. Drag your white cursor quickly across the screen against a black background. You will see clear gaps between the individual cursor afterimages on your retina. That's the outcome of aliasing, not of the FPS limitation itself. You could analytically supersample the motion in the time domain and then blur it just enough to remove the aliasing, and the distinct images would then disappear. Motion blur approximates the same result.
  - blauditore 3y ago
    That's not true at all. When moving multiple pixels per frame, there will always be visible jumps. Aliasing only becomes relevant at the pixel level.
    - zozbot234 3y ago
      "Pixels" and "frames" are the exact same thing analytically, only in the spatial vs. time domain. This is very much an instance of aliasing, which is why blur happens to correct it.
      - blauditore 3y ago
        Alright, but supersampling would exactly achieve another approximation to (real) motion blur, but still suffer from the same issues (like not becoming sharp when tracking with your eyes).
harimau777 3y ago
Can you recommend any resources on how shutter angle is used to communicate different messages in film? The Wikipedia article only gives some very basic examples (short shutter angle when you want to capture particles in the air or for action scenes).
- chaboud 3y ago
  https://www.indepthcine.com/videos/shutter-angle https://www.indepthcine.com/videos/shutter-angle

pierrec 3y ago

Exactly, while doing background research for this I read a really neat paper that describes what you're saying, and offers a solution based on trying to predict the viewer's eye motion: Temporal Video Filtering and Exposure Control for Perceptual Motion Blur (Stengel et al., 2015)

Veedrac 3y ago

You can't properly make a moving image sharp at a finite refresh rate (though you can approach it for sufficiently high ones), because the object can only move in a temporally juddered path, which doesn't match the eye's movement.

planede 3y ago
You can with a strobed backlight.
- Veedrac 3y ago
  Fair point.

nyanpasu64 3y ago

Moving objects can become even sharper if each frame is displayed in a fixed location for a shorter period of time (reduced MPRT), preventing eye-tracking motion from smearing each displayed frame. This can be achieved through CRT/OLED scanout (often rolling) or LCD backlight strobing (usually full-frame by necessity). Unfortunately displaying each frame for a short time is unbearably flickery at 24 Hz (so movie projectors would show film frames 2-3 times), just barely tolerable at 50 (to the point some European CRT televisions frame-doubled 50 Hz video to 100 Hz, causing objects to appear doubled in motion), and ideally needs 70-75 Hz or above for optimally fluid motion and minimum eyestrain (which can't show 60 FPS recorded video without judder and/or tearing).

s4i 3y ago
> to the point some European CRT televisions frame-doubled 50 Hz video to 100 Hz, causing objects to appear doubled in motion Hmm, this is not accurate (or I don't understand what you mean). 100Hz CRT TVs available in the 90s/00s did not interpolate frames to get smoother motion – they only existed to reduce flicker. I think such TVs also existed in NTSC markets (120Hz)? Anyway, ever since the late 00s, pretty much all the TVs you can buy from a store do come with an interpolation algorithm to artificially display a higher frame rate image (e.g. 100Hz/120Hz) from a lower frame rate source (e.g. 23.976/24/29.97/30/50/59.94/60 fps) – which (personal opinion) looks terrible (and can be turned off from the settings – but the default is always on). This is an interesting side tangent when it comes to motion blur, because the blur is prebaked in the input signal and cannot be easily removed. Thus, the end result always has an artificial look. For instance, if the source material is shot in 24fps with the typical 180 degree shutter angle, each frame spans 41.6ms of which the shutter was open for 20.8ms. Then your TV interpolates that to be 96Hz or whatever. However, the individual output frames still look (or, with the added artifacts etc., mostly look) like the shutter was open for 20.8ms per frame. However, each frame now spans 10.2ms which is a shorter time than the shutter speed!
- cubefox 3y ago
  As far as I understand, you get doubling not because of any interpolation but because you track objects with your eye. That means the tracked object is in rest in your vision. Which means the screen actually moves relative to your vision. Which means the object smears, where the pixel length of the smear is equal to the pixel distance the tracked object moves in the time the frame is visible on screen. On CRT there is less smear, because the frames are only flashed for a short time on the otherwise dark screen. But if the FPS are not equal to the display Hz, e.g. a half, the tracked object appears on multiple, e.g. two, locations on the screen. Because you show the frame twice, and if you track an object, the screen has moved (relative to your vision) for a few pixels by the time you show the second frame. This happens e.g. if you play a 30 FPS game on a 60 Hz CRT, or when you watch 50 FPS content on a 100 Hz screen. Though the effect isn't usually as dramatic as it may sound here. Doubling is also not as bad as smearing, e.g. in the case of scrolling text.

cubefox 3y ago

An alternative to infinite refresh rate is to show each frame only a fraction of the normal frame time, i.e. to quickly "flash" the individual frames and showing a black screen otherwise. This reduces the maximum screen brightness, an it requires a minimum frame rate to avoid flicker, but it reduces the type of "eye tracking blur" (persistence blur) which isn't present in real life. To be precise, to completely remove the tracking blur you would need to flash each frame only for an infinitesimal time period, which of course isn't realistic. But VR headsets do use this flashing/strobing approach. By the way, this is also the reason why CRT and Plasma screens had much better motion clarity than LCD or OLED. The former flash each frame for a short time, while the latter "sample and hold" the frame for the entire frame time (e.g. 1/60th of a second for 60 Hz). 60 FPS on a CRT looks probably more fluid than 120 FPS on an OLED. Another option for games is to indeed add a lot of frames by using reprojection techniques. This can approximate the real camera movements without needing to render a ton of expensive frames in the engine. This also is already used in VR, just currently not at overly high frame rates. This great article goes into more detail: https://blurbusters.com/frame-generation-essentials-interpolation-extrapolation-and-reprojection/ https://blurbusters.com/frame-generation-essentials-interpol... Something like 1000 FPS with reprojection are apparently quite realistic, which should solve the problem of tracking blur without reducing screen brightness.

account42 3y ago
> Something like 1000 FPS with reprojection are apparently quite realistic, which should solve the problem of tracking blur without reducing screen brightness. Is a 1000 FPS screen more realistic than a screen capable of the higher maximum brightness needed to compensate for black frame insertion though? HDR screens are already a thing and you could already gain persistence improvements there for LDR scenes without needing any new hardware by always driving pixels at max brighness but only for a reduced time depending on the target luminance. Or just reduce the ambient light enough - I even run even my LDR monitor at 10% brightness.
- cubefox 3y ago
  As far as I understand, the only thing that matters for reducing unrealistic "tracking blur" is indeed how short a frame is displayed on screen. Which could be achieved by strobing / black frame insertion or by increasing frame rate. Or even both, as in some VR applications. So the effect should be the same, except for the brightness thing, which may not be a problem if there is a lot of screen brightness headroom anyway. For HDR LCD TVs the LED backlights get quite bright actually. OLED not so much. One advantage of higher frame rates (as opposed to strobing) would be quicker input responses to, e.g., moving the camera. That's not overly important on a normal screen but quite significant on a VR headset where we expect head movements to be represented very quickly.