6 ms·

It seems like we are coming full circle in try to reduce latency in all parts of the chain. With the next gen gaming consoles supporting VRR and 120hz, recently

It seems like we are coming full circle in try to reduce latency in all parts of the chain. With the next gen gaming consoles supporting VRR and 120hz, recently announced Nvidia Reflex low latency modes, 360hz monitors, and things like the Apple Pencil 2 reducing the latency from pen input to on screen down to 9ms we are working our way back to something close to what we used to have. But I feel like we have a number of years to go until we can really get back to where we used to be with vintage gaming consoles and CRT displays.

perl4ever 6y ago
I think that with the end of Moore's law, there is a gradual unwinding of software bloat and inefficiency. Once the easy hardware gains die away, it is worth studying how software can be better, but it's a cultural transition that takes time.
- stephc_int13 6y ago
  I also think that the end of the so called Moore Law will create the necessary incentives to build better software.
  - ginko 6y ago
    What's with the "so called"? Moore's Law is a pretty well-established term.
    - stephc_int13 6y ago
      It is not a law, it is merely an observation.
- jhayward 6y ago
  It is also the case that in some/many areas software has improved performance by several multiples of the improvement in hardware performance over the last 20 years. So it makes sense that this would invigorate software performance investment.
  - mpweiher 6y ago
    Can you give some examples of areas where software performance improvement has been several multiples of hardware improvement? Most of the examples I can think of are ones where the software slowdown has more than cancelled out the hardware improvements. Then there are a some areas where hardware performance improvement was sufficient to overcome software slowdown. Software getting faster? Software getting faster than hardware??
    - MayeulC 6y ago
      Compilers are way smarter and can make old code faster than it used to run. They also parse the code way faster, and would as such compile faster if you restrict them to the level of optimizations they used to do. "Interpreters" are faster as well: Lisp runs faster, JavaScript is orders of magnitudes more efficient as well. Algorithms have been refined. Faster paths have been uncovered for matrix multiplication [1] (unsure if the latest improvements are leveraged) and other algorithms. Use-cases that have been around fo run a wile (say, h.264 encode/decode) are more optimized. We now tend to be a lot better at managing concurrency too (see: rust, openmp and others), with the massively parallel architectures that come out nowadays. [1] https://en.m.wikipedia.org/wiki/Matrix_multiplication_algorithm https://en.m.wikipedia.org/wiki/Matrix_multiplication_algori...
      - mpweiher 6y ago
        Thanks for the examples! However, I can't really agree that they are examples of software improvements being multiples of hardware improvements. 1. Compiler optimizations See Proebsting's Law [1], which states that whereas Moore's Law provided a doubling of performance ever 18-24 months, compiler optimizations provide a doubling every 18 years at best. More recent measurements indicate that this was optimistic. 2. Compilers getting faster Sorry, not seeing it. Swift, for example, can take a minute before giving up on a one line expression, and has been clocked at 16 lines/second for some codebases. All the while producing slow code. See also part of the motivation for Jonathan Blow's Jai programming language. 3. Matrix multiplication No numbers given, so ¯\_(ツ)_/¯ 4. h.264/h.265 The big improvements have come from moving them to hardware. 5. Concurrency That's hardware. [1] https://www.semanticscholar.org/paper/On-Proebsting%27%27s-Law-Scott/0a2b1aa8bb63fb545f7f41233e5d6c0206486ccc?p2df https://www.semanticscholar.org/paper/On-Proebsting%27%27s-L...
ajconway 6y ago
It seems to be a normal engineering process: first make it work, then look for those high-level things that you are ready to break in order to gain performance through optimization and micro-optimization.
MayeulC 6y ago
> But I feel like we have a number of years to go until we can really get back to where we used to be with vintage gaming consoles and CRT displays. Interesting. CRTs also had a fixed framerate. Let's make that 60 fps for the sake of the argument. It really depends on what you are calling "vintage". Most later consoles (with GPUs) just composited images at a fixed frame-rate. Earlier software renderers are quite interesting, though, in that they tend to "race the beam" and produce pixel data a few microseconds before it is displayed. Does that automatically transfer to low latency? I'm not sure. If the on-screen character is supposed to move by a few pixels with the last input, it really depends on whether you have drawn it already. Max latency is 16 ms, min is probably around 100 µs. That gives you 8 ms of expected latency. And I think you still get tearing, in some cases. There is also no reason it couldn't be done with modern hardware, except the wire data format for HDMI/DP might need to be adjusted. However, and I've said that for a long time, one key visible difference between CRTs and LCDs is persistence. Images on a LCD persist for a full frame, instead of letting your brain interpolate the images. The result is a distinctively blurry edge on moving objects. Some technologies such as backlight strobing (aka ULMB) aim to aleviate this (you likely need triple buffering to combine this with adaptative sync, which I haven't seen). I wonder if rolling backlights could allow us to race the beam once again? QLED/OLED displays could theoretically bring a better experience than CRTs if the display controller allowed it: every pixel emits its own light, so low persistence is achievable. You don't have a beam with fixed timings, so you could just update what's needed in time for displaying it.
- dan-robertson 6y ago
  A separate latency difference is in how long it takes for the pixels to switch. Ie the time between when the device starts trying to change the colour of a pixel and when you see that colour change. This is a relatively long time for an lcd but not so long for a crt
  - MayeulC 6y ago
    Well, isn't it an aspect of persistence? This is usually called ghosting, and can be combated using "overdrive"; https://blurbusters.com/faq/oled-motion-blur/ https://blurbusters.com/faq/oled-motion-blur/ https://blurbusters.com/faq/lcd-overdrive-artifacts/ https://blurbusters.com/faq/lcd-overdrive-artifacts/ Edit: regarding my older comment, I thought that QLED were quantum dots mounted on individual LEDs. They are regular LCDs, with quantum dots providing the colour conversion. That makes more sense from an economical perspective, less so for performance. Maybe OLED and QLED could be combined to leverage the best OLED for all colors?

perl4ever 6y ago

I think that with the end of Moore's law, there is a gradual unwinding of software bloat and inefficiency. Once the easy hardware gains die away, it is worth studying how software can be better, but it's a cultural transition that takes time.

stephc_int13 6y ago
I also think that the end of the so called Moore Law will create the necessary incentives to build better software.
- ginko 6y ago
  What's with the "so called"? Moore's Law is a pretty well-established term.
  - stephc_int13 6y ago
    It is not a law, it is merely an observation.
jhayward 6y ago
It is also the case that in some/many areas software has improved performance by several multiples of the improvement in hardware performance over the last 20 years. So it makes sense that this would invigorate software performance investment.
- mpweiher 6y ago
  Can you give some examples of areas where software performance improvement has been several multiples of hardware improvement? Most of the examples I can think of are ones where the software slowdown has more than cancelled out the hardware improvements. Then there are a some areas where hardware performance improvement was sufficient to overcome software slowdown. Software getting faster? Software getting faster than hardware??
  - MayeulC 6y ago
    Compilers are way smarter and can make old code faster than it used to run. They also parse the code way faster, and would as such compile faster if you restrict them to the level of optimizations they used to do. "Interpreters" are faster as well: Lisp runs faster, JavaScript is orders of magnitudes more efficient as well. Algorithms have been refined. Faster paths have been uncovered for matrix multiplication [1] (unsure if the latest improvements are leveraged) and other algorithms. Use-cases that have been around fo run a wile (say, h.264 encode/decode) are more optimized. We now tend to be a lot better at managing concurrency too (see: rust, openmp and others), with the massively parallel architectures that come out nowadays. [1] https://en.m.wikipedia.org/wiki/Matrix_multiplication_algorithm https://en.m.wikipedia.org/wiki/Matrix_multiplication_algori...
    - mpweiher 6y ago
      Thanks for the examples! However, I can't really agree that they are examples of software improvements being multiples of hardware improvements. 1. Compiler optimizations See Proebsting's Law [1], which states that whereas Moore's Law provided a doubling of performance ever 18-24 months, compiler optimizations provide a doubling every 18 years at best. More recent measurements indicate that this was optimistic. 2. Compilers getting faster Sorry, not seeing it. Swift, for example, can take a minute before giving up on a one line expression, and has been clocked at 16 lines/second for some codebases. All the while producing slow code. See also part of the motivation for Jonathan Blow's Jai programming language. 3. Matrix multiplication No numbers given, so ¯\_(ツ)_/¯ 4. h.264/h.265 The big improvements have come from moving them to hardware. 5. Concurrency That's hardware. [1] https://www.semanticscholar.org/paper/On-Proebsting%27%27s-Law-Scott/0a2b1aa8bb63fb545f7f41233e5d6c0206486ccc?p2df https://www.semanticscholar.org/paper/On-Proebsting%27%27s-L...

ajconway 6y ago

It seems to be a normal engineering process: first make it work, then look for those high-level things that you are ready to break in order to gain performance through optimization and micro-optimization.

MayeulC 6y ago

> But I feel like we have a number of years to go until we can really get back to where we used to be with vintage gaming consoles and CRT displays. Interesting. CRTs also had a fixed framerate. Let's make that 60 fps for the sake of the argument. It really depends on what you are calling "vintage". Most later consoles (with GPUs) just composited images at a fixed frame-rate. Earlier software renderers are quite interesting, though, in that they tend to "race the beam" and produce pixel data a few microseconds before it is displayed. Does that automatically transfer to low latency? I'm not sure. If the on-screen character is supposed to move by a few pixels with the last input, it really depends on whether you have drawn it already. Max latency is 16 ms, min is probably around 100 µs. That gives you 8 ms of expected latency. And I think you still get tearing, in some cases. There is also no reason it couldn't be done with modern hardware, except the wire data format for HDMI/DP might need to be adjusted. However, and I've said that for a long time, one key visible difference between CRTs and LCDs is persistence. Images on a LCD persist for a full frame, instead of letting your brain interpolate the images. The result is a distinctively blurry edge on moving objects. Some technologies such as backlight strobing (aka ULMB) aim to aleviate this (you likely need triple buffering to combine this with adaptative sync, which I haven't seen). I wonder if rolling backlights could allow us to race the beam once again? QLED/OLED displays could theoretically bring a better experience than CRTs if the display controller allowed it: every pixel emits its own light, so low persistence is achievable. You don't have a beam with fixed timings, so you could just update what's needed in time for displaying it.

dan-robertson 6y ago
A separate latency difference is in how long it takes for the pixels to switch. Ie the time between when the device starts trying to change the colour of a pixel and when you see that colour change. This is a relatively long time for an lcd but not so long for a crt
- MayeulC 6y ago
  Well, isn't it an aspect of persistence? This is usually called ghosting, and can be combated using "overdrive"; https://blurbusters.com/faq/oled-motion-blur/ https://blurbusters.com/faq/oled-motion-blur/ https://blurbusters.com/faq/lcd-overdrive-artifacts/ https://blurbusters.com/faq/lcd-overdrive-artifacts/ Edit: regarding my older comment, I thought that QLED were quantum dots mounted on individual LEDs. They are regular LCDs, with quantum dots providing the colour conversion. That makes more sense from an economical perspective, less so for performance. Maybe OLED and QLED could be combined to leverage the best OLED for all colors?