17 ms·

I love a small feature he mentioned in the video: Monitors with built-in latency measurement. You plug your mouse into the USB ports on the monitor and it tells

I love a small feature he mentioned in the video: Monitors with built-in latency measurement. You plug your mouse into the USB ports on the monitor and it tells you how long it takes the image to change after you move the mouse. Brilliant idea. It's long overdue to have widely available metrics for latency in consumer tech. Many hardware/software setups have absurdly long latency for no good reason other than that it's difficult to measure. People underestimate the subconscious effects it has on your usage of technology. I couldn't be happier that phone and monitor manufacturers are finally starting to compete on refresh rates >60 Hz. It's far more important than wide gamut or 8K.

seiferteric 6y ago
I wish monitors in general had better (and open) firmware. It would be really cool if for example I could have my laptop plugged into my desktop's monitor and have it show up as a "window" on my desktop. All it would take would be for some simple driver on my desktop to tell the monitor where to draw the image when I moved the window around. Basically a better version of the mostly useless PiP feature my monitor already has.
- slimsag 6y ago
  As with many things, this comes back to DRM. HDCP in specific.
  - freeone3000 6y ago
    Monitors are generally connected through DisplayPort or Thunderbolt, not HDMI, so there's no reason for HDCP to enter here. The card in question does have a HDCP output, but advanced features are only available through DisplayPort.
    - nailer 6y ago
      > Monitors are generally connected through DisplayPort or Thunderbolt, not HDMI, so there's no reason for HDCP to enter here. HDCP runs over DP and Thunderbolt. But derefr's discussion above is accurate.
    - ChuckNorris89 6y ago
      It might surprise you that most monitors out there, especially the cheap garbage ones from Amazon/Walmart that most consumers have in their homes, are connected via HDMI.
      - tgb 6y ago
        As someone running their monitor over HDMI, this is news to me. If it gives the resolution and framerate of the monitor, what's the difference if it's HDMI or DisplayPort?
        ChuckNorris89 6y ago
        For you as an end consumer nothing if bandwidth is enough with either cabke, but techies like us will be sticklers that one is more open than the other, not bound by royalties and such.
        formerly_proven 6y ago
        Displayport usually has something like 50 % higher bandwidth compared to the HDMI standard of the same vintage. Which makes sense, since home video folks are fine with like 15 pictures per second and 4:0:0 chroma subsampling, but that doesn't really work for computers...
        jasomill 6y ago
        That, and some video cards support higher resolutions over DisplayPort. For example, while my 2013 Mac Pro has both HDMI and Thuderbolt/Mini DisplayPort outputs, the only way to get 2160p60 HDMI output (without chroma subsampling) is via a DisplayPort-to-HDMI 2.x adapter (and I had to try about half a dozen different adapters before I found one that does this correctly).
        Polylactic_acid 6y ago
        Displayport used to be much better but now they both cover virtually everyones needs. Now the main difference is HDMI has a licensing fee. Some major TV companies are in the org so they don't have to pay it which is why TVs always use HDMI and GPUs usually have more displayport because those companies have to pay for hdmi.
  - derefr 6y ago
    In this particular case, no, as the image wouldn't be being sent into the PC to render in its framebuffer; but rather the PC would just be drawing an empty window, and reporting the geometry of that window to the monitor. The monitor, with two HDMI leads plugged in, would be responsible for compositing the inputs together, according to the geometry the PC reported, but all internal to itself.
    - smileybarry 6y ago
      That's how hardware-accelerated video decoding used to work in Windows XP days IIRC (before GPU-based desktop composition), the video player would be a blank black square and the GPU would be told to draw the video on those coordinates. Because of how it was implemented, you could drag VLC around while the video was playing and the video would stay "behind" everything, with the VLC window acting as a "hole" through which you could see it. (So you could move the window to the left and see half a black square on the left, and the left-most half of the video on the right) Nowadays with desktop composition AKA DWM, Windows just makes sure to black out DRM content from any frames/video it sends to an app requesting to capture the screen, making sure to send the video-including composed desktop only to the display. (And if you have some GPU recording software like NVIDIA ShadowPlay, it switches off when DRM content starts playing) You can see it in action with the Netflix UWP app. Of course, a bunch of DRM-respecting software -- like Netflix in Google Chrome -- doesn't really follow that spec and can still be screenshot/video captured like any app.
      - judge2020 6y ago
        I found it odd that even Disney Plus - which I heard had the highest level security of Widevine enabled, can be screen recorded.
        Mindwipe 6y ago
        It provides lower resolution content to those devices, capping at 720p. Which is the same as Netflix effectively, Netflix just provides higher to PCs when they can block screen recording entirely. Disney might eventually I guess.
- nradov 6y ago
  Just use remote control software to connect from one computer to the other and display it as a window.
  - seiferteric 6y ago
    I do, but now you have latency.
    - aaronblohowiak 6y ago
      HDMI encoder will be better than remote software, in my experience (depending on the hdmi encoder)
- Gehinnn 6y ago
  It would be cool if the entire Onscreen Menu would be available as api. I hate using it to switch between inputs or to turn of my second monitor.
  - MaPi_ 6y ago
    Look into DCC/CI, it's possible your monitor supports that already.
ponker 6y ago
> It's far more important than wide gamut or 8K. This is just a matter of opinion. Wide gamut is more important to me, latency is more important to you.
- hyko 6y ago
  ...and I’d opt for the spatial resolution, completing the trifecta! Thankfully, we can just have all three :)
  - formerly_proven 6y ago
    > Thankfully, we can just have all three :) Not at this time. There are some hypothetical options like the LG 27GN950, but it has poor contrast and uses Displayport 1.4 compression, which is only supported in the latest graphics cards. VESA and friends really made DisplayPort 1.4 as close as they possibly could to false advertising without actually committing it (because the only relevant new thing in DP 1.4 -- DSC, display stream compression -- is technically optional and no one claimed they'd support DSC while saying they support DP 1.4, which is pretty much the same as supporting DP 1.3, since nothing of note changed). And now we're looking at DisplayPort 2.0 which can already barely support uncompressed 8K at 60 Hz and is basically maxed out by 5K/6K 120/144 Hz. And it's unclear if the presently introduced generation of GPUs even supports it, or if we're going to use effectively-2013-Displayport until ~2022. Note how the marketing material only talks about HDMI 2.1; DisplayPort isn't mentioned once.
    - tormeh 6y ago
      I think DisplayPort is not mentioned because Nvidia hasn't upgraded their cards to 2.0. Super disappointing. 4k/144Hz/HDR is enough for now, to be honest, but DP2 can do 84/144Hz/HDR as long as it has DSC, 244Hz with subsampling. The slow pace of development and deployment has hurt everyone for sure, but I don't think the standard is bad in itself - just a bit late.
    - hyko 6y ago
      Yeah, good points–it seems even armed with a 3090 we'll be limited to 8K/120hz/12 bit over HDMI 2.1...we'll have to wait a while before we're actually driving 8K at higher refresh rates.
TaylorAlexander 6y ago
Good thoughts all around. I will say that I am excited about 8k video for computational photography. I am studying computer vision for robotics and it is quite clear to me that a simple spherical lens with a very high resolution sensor would be a very good imaging system for a real world robot. I recently got a Panasonic GH5 and it shoots 18 megapixel “6k” 30fps video, encoded in h265 (important for higher res). I am experimenting with photogrammetry using this photo mode. The initial results are very promising. In four minutes I took a scan of a short street block in Oakland, and after processing the photogrammetry solution I have a very good dense 3D full color map of the street. The model has holes but I am slowly learning how to plan a photogrammetry capture. Currently computation takes hours but I am seeing more and more research on ways that machine learning can improve compute times and data density. See how low the visual acuity is on 5.6k spherical video here: https://youtu.be/nASvIYq3VkE https://youtu.be/nASvIYq3VkE However all this is to say that very high resolution sensors are a very good thing for robotics.
- strogonoff 6y ago
  Could it be faster to skip in-camera encoding and build a 3D scene based on raw scene-referred data?
  - modeless 6y ago
    I think the far future of computer vision will be "event cameras" feeding directly into ML systems: https://en.wikipedia.org/wiki/Event_camera https://en.wikipedia.org/wiki/Event_camera
  - namibj 6y ago
    It's not efficient to do all of this live. Skipping the H.265 parts would likely be beneficial, though.
- namibj 6y ago
  Hi there! So, all free/open multi-view stereo and structure-from-motion software I know of is incapable of handling rolling shutter artifacts. The problem seems to be that electronic global shutter sensors lack (some) dynamic range compared to otherwise-equal rolling shutter cameras. If you'd be interested in talking more about this, contact me/let me know (I'll get in touch if requested). My photogrammetry experiments typically encompass low-effort bulk data collection, though the search for a light-weight camera to use with a prosumer-class drone and some revelations about reconstruction quality issues inherent to older, easily-optimized algorithms for both multi-view stereo and mesh reconstruction stalled progress somewhat. In general, machine learning doesn't seem to be as much of a benefit as one might guess, when compared to applying the resources in non-ML ways to the data. Mind teasing some numbers from your street capture?
jacquesm 6y ago
That's because our operating systems treat user input as a soft real time problem when actually it should be a hard real time problem. That's why your window manager sometimes can be unresponsive. I've used a hard real time OS for a desktop for some years and the difference with consumer stuff was extreme. You get used to the computer instantly doing what you tell it to do. None of these half second or longer delays between command and result. It's like magic.
- burlesona 6y ago
  Which OS? What is your setup?
  - deaddodo 6y ago
    Probably QNX or VxWorks.
    - neilpanchal 6y ago
      Also RTLinux kernel: https://en.wikipedia.org/wiki/RTLinux https://en.wikipedia.org/wiki/RTLinux
      - monocasa 6y ago
        RTLinux is soft real time, FWIW.
        rudolph9 6y ago
        Can you elaborate? The wiki says it combines hard-realtime with soft-realtime. It sounds like it depends how you configure it but this is the first I’ve read of it so I could be mistaken.
        kijiki 6y ago
        They're just mistaken, RTLinux is hard realtime. It works by running a tiny real time OS on the bare metal, and then running a Linux kernel as a (lightly) paravirtualized lowest priority task. You write your hard realtime code to run on the RTOS, and then your soft/non realtime code runs on Linux. In this model, your compositor, input pipeline and (at least the UI latency sensitive part of) your applications would have to be ported to the RTOS, which makes this pretty infeasible. But it works really well if you have a some hard-realtime control loop talking to a bunch of non-realtime network IO or UI code. Would be a fun way to build a wifi controlled quadcopter. These days you probably want to use Xenomai, or possibly RTAI.
nickjj 6y ago
Fortunately a number of monitor review sites include input latency that's properly measured. For example https://www.tftcentral.co.uk/ https://www.tftcentral.co.uk/. They have a lot of popular models and super in depth reviews.
- tgtweak 6y ago
  Rtings pretty solid too
  - zmix 6y ago
    And, should one speak German, there is https://www.prad.de/ https://www.prad.de/ too.
nomel 6y ago
> You plug your mouse into the USB ports on the monitor and it tells you how long it takes the image to change after you move the mouse. Brilliant idea. Related, "Your mouse is a terrible webcam", 2014: https://hackaday.com/2014/01/14/your-mouse-is-a-terrible-webcam/ https://hackaday.com/2014/01/14/your-mouse-is-a-terrible-web...
dan-robertson 6y ago
This does encourage optimising the OS or application part (eg double buffering and taking another frame is slow), but it doesn’t measure two significant sources of latency: The latency in the hardware before the signal makes it to the usb (for most keyboards, even specialised gaming keyboards this is like 30ms, which is 2 frames). There is also the latency from when the monitor gets a signal to when the pixels have perceptually finished transitioning. This can be 10s of ms too. So even if the OS has 0 latency, and the monitor measures that, you could still easily observe a latency of say 60ms.
- modeless 6y ago
  True! I just watched this video [1] they put out with more details, and it sounds like they are working with monitor and mouse manufacturers to build true end-to-end latency measurement when using supported hardware. It would be easy for manufacturers to game these numbers, but hopefully Nvidia will provide some enforcement to prevent that. Maybe they could also add a microphone to measure audio latency because that is another huge problem with modern systems. The video mentions that they get 15 ms end-to-end latency on a 360 Hz monitor in Fortnite. So that's what it takes for a modern system to finally beat the latency that you used to get on, say, an NES connected to a CRT. Still an order of magnitude above the limits of perception though [2]. [1] https://www.youtube.com/watch?v=-cXg7GQogAE https://www.youtube.com/watch?v=-cXg7GQogAE [2] https://www.youtube.com/watch?v=vOvQCPLkPt4 https://www.youtube.com/watch?v=vOvQCPLkPt4
  - ksec 6y ago
    Yes! I watched that video many years ago. And then later I remember when media were hyping how "responsive" Apple Pencil was I was thinking what on earth are these people looking at. I thought to myself we would never get there because no one gives a damn about it. How the world has changed! I often wonder at what cost, could we get it down to Sub Ms. Where everything from Input, Video and Audio are sync to the precision of 1ms. There used to be very little incentive to do so. Now that Gaming has taken over I hope it will develop some market perception and value so we could finally move into that direction. The age of Instantaneous Computing.
- 6y ago
jayd16 6y ago
Its not really full end to end latency though. I think, at best it can measure when the the monitor thinks it received the signal from the mouse and when it perceives the new image. Latency in the monitor itself can't be measured, can it?
- nightski 6y ago
  Is there any reason to believe latency within the monitor itself wouldn't be relatively constant? Or would you expect large variances in monitor latency? If it was a pretty narrow distribution they could simply add on that constant factor. I'm not sure if they do this though.
ksec 6y ago
And I am surprised [1] Link hasn't pop up yet. Yes. The industry has been optimising for throughput in the past decades. It is time to bring latency back to the table. I want super low latency Computing. Street Fighter or The King of Fighters in CRT Arcade Era just felt so much more responsive. https://danluu.com/input-lag/ https://danluu.com/input-lag/
- CarVac 6y ago
  Using Project Slippi, you can play Super Smash Bros Melee at the same latency as on a CRT… over the internet! (Melee on a CRT has 3 frames of input lag)
amelius 6y ago
> Monitors with built-in latency measurement. They should start measuring the time it takes for monitors to recognize the signal when you press the "Source" button. Even on recent monitors, it's often 5 seconds or more.
- modeless 6y ago
  Yes! I would love for more review sites to benchmark this, as well as time to wake from sleep. I have a gsync monitor that has only one input port and it wakes from sleep in like 0.2 seconds. After the experience of using it for a while, if I now had to choose between gsync and fast wake from sleep I would choose fast wake from sleep every time.
- SnowflakeOnIce 6y ago
  My 2019 Dell monitors take 15s :|
- Unklejoe 6y ago
  Part of this is because of the complex equalization that must be performed when locking to the incoming signal. Basically, the device has to sweep through several different EQ settings until it achieves the combination with the best signal eye. You'd think this would be lightning fast, but as it turns out, this is often handled in firmware (with hardware support for measuring the eye). I've seen this process take upwards of two seconds. A lot of these displays probably only have one RX core, so it's not like it can remain locked to all inputs at the same time.
skohan 6y ago
In addition to input latency, with "real time" computer graphics it's basically impossible to know when your frame will actually be rendered on screen. If this information was available, it would be a lot easier to achieve smooth visuals without having to chase higher and higher frame-rates.
- vlovich123 6y ago
  Don’t GPU fences tell you which vsync you’re targeting? Or are you saying you want the actual estimated display time of the frame? I know the latter is definitely available for VR systems as it’s required. Having it available for 2d games probably would help marginally in cutting down the perceived latency although I don’t know if you’d be able to feel it.
  - sp332 6y ago
    John Carmack has mentioned on Twitter that he spent a lot of time trying to make the frame release rate more consistent even with purpose-built VR hardware. I think game programmers found it easier to go with the gsync/freesync method of doing a vsync whenever the frame is ready instead of trying to hit a deadline.
  - skohan 6y ago
    Yes I want the actual estimated display time. This would allow for more accurate interpolation.
mesh 6y ago
Here is their video on it: https://www.youtube.com/watch?v=-cXg7GQogAE https://www.youtube.com/watch?v=-cXg7GQogAE
mensetmanusman 6y ago
Don’t forget NITS. With a high NITS display, we could read outside in the bright sunlight during a pandemic.
- DiabloD3 6y ago
  Nits is not to be capitalized.
  - mensetmanusman 6y ago
    I always yell NITS
trilinearnz 6y ago
I recall John Carmack being interested in measuring latency in exactly this way during his Occulus tenure (think he mentioned it in one of his keynotes). I'm sure he would be pleased to see this is now becoming mainstream :)
- ddalex 6y ago
  I did it at Intel for their Android tablets (a decade ago) using a high framerate (120Hz) webcam and counting frames. A bit cumbersome but effective.

seiferteric 6y ago

I wish monitors in general had better (and open) firmware. It would be really cool if for example I could have my laptop plugged into my desktop's monitor and have it show up as a "window" on my desktop. All it would take would be for some simple driver on my desktop to tell the monitor where to draw the image when I moved the window around. Basically a better version of the mostly useless PiP feature my monitor already has.

slimsag 6y ago
As with many things, this comes back to DRM. HDCP in specific.
- freeone3000 6y ago
  Monitors are generally connected through DisplayPort or Thunderbolt, not HDMI, so there's no reason for HDCP to enter here. The card in question does have a HDCP output, but advanced features are only available through DisplayPort.
  - nailer 6y ago
    > Monitors are generally connected through DisplayPort or Thunderbolt, not HDMI, so there's no reason for HDCP to enter here. HDCP runs over DP and Thunderbolt. But derefr's discussion above is accurate.
  - ChuckNorris89 6y ago
    It might surprise you that most monitors out there, especially the cheap garbage ones from Amazon/Walmart that most consumers have in their homes, are connected via HDMI.
    - tgb 6y ago
      As someone running their monitor over HDMI, this is news to me. If it gives the resolution and framerate of the monitor, what's the difference if it's HDMI or DisplayPort?
      - ChuckNorris89 6y ago
        For you as an end consumer nothing if bandwidth is enough with either cabke, but techies like us will be sticklers that one is more open than the other, not bound by royalties and such.
        formerly_proven 6y ago
        Displayport usually has something like 50 % higher bandwidth compared to the HDMI standard of the same vintage. Which makes sense, since home video folks are fine with like 15 pictures per second and 4:0:0 chroma subsampling, but that doesn't really work for computers...
        jasomill 6y ago
        That, and some video cards support higher resolutions over DisplayPort. For example, while my 2013 Mac Pro has both HDMI and Thuderbolt/Mini DisplayPort outputs, the only way to get 2160p60 HDMI output (without chroma subsampling) is via a DisplayPort-to-HDMI 2.x adapter (and I had to try about half a dozen different adapters before I found one that does this correctly).
      - Polylactic_acid 6y ago
        Displayport used to be much better but now they both cover virtually everyones needs. Now the main difference is HDMI has a licensing fee. Some major TV companies are in the org so they don't have to pay it which is why TVs always use HDMI and GPUs usually have more displayport because those companies have to pay for hdmi.
- derefr 6y ago
  In this particular case, no, as the image wouldn't be being sent into the PC to render in its framebuffer; but rather the PC would just be drawing an empty window, and reporting the geometry of that window to the monitor. The monitor, with two HDMI leads plugged in, would be responsible for compositing the inputs together, according to the geometry the PC reported, but all internal to itself.
  - smileybarry 6y ago
    That's how hardware-accelerated video decoding used to work in Windows XP days IIRC (before GPU-based desktop composition), the video player would be a blank black square and the GPU would be told to draw the video on those coordinates. Because of how it was implemented, you could drag VLC around while the video was playing and the video would stay "behind" everything, with the VLC window acting as a "hole" through which you could see it. (So you could move the window to the left and see half a black square on the left, and the left-most half of the video on the right) Nowadays with desktop composition AKA DWM, Windows just makes sure to black out DRM content from any frames/video it sends to an app requesting to capture the screen, making sure to send the video-including composed desktop only to the display. (And if you have some GPU recording software like NVIDIA ShadowPlay, it switches off when DRM content starts playing) You can see it in action with the Netflix UWP app. Of course, a bunch of DRM-respecting software -- like Netflix in Google Chrome -- doesn't really follow that spec and can still be screenshot/video captured like any app.
    - judge2020 6y ago
      I found it odd that even Disney Plus - which I heard had the highest level security of Widevine enabled, can be screen recorded.
      - Mindwipe 6y ago
        It provides lower resolution content to those devices, capping at 720p. Which is the same as Netflix effectively, Netflix just provides higher to PCs when they can block screen recording entirely. Disney might eventually I guess.
nradov 6y ago
Just use remote control software to connect from one computer to the other and display it as a window.
- seiferteric 6y ago
  I do, but now you have latency.
  - aaronblohowiak 6y ago
    HDMI encoder will be better than remote software, in my experience (depending on the hdmi encoder)
Gehinnn 6y ago
It would be cool if the entire Onscreen Menu would be available as api. I hate using it to switch between inputs or to turn of my second monitor.
- MaPi_ 6y ago
  Look into DCC/CI, it's possible your monitor supports that already.

ponker 6y ago

> It's far more important than wide gamut or 8K. This is just a matter of opinion. Wide gamut is more important to me, latency is more important to you.

hyko 6y ago
...and I’d opt for the spatial resolution, completing the trifecta! Thankfully, we can just have all three :)
- formerly_proven 6y ago
  > Thankfully, we can just have all three :) Not at this time. There are some hypothetical options like the LG 27GN950, but it has poor contrast and uses Displayport 1.4 compression, which is only supported in the latest graphics cards. VESA and friends really made DisplayPort 1.4 as close as they possibly could to false advertising without actually committing it (because the only relevant new thing in DP 1.4 -- DSC, display stream compression -- is technically optional and no one claimed they'd support DSC while saying they support DP 1.4, which is pretty much the same as supporting DP 1.3, since nothing of note changed). And now we're looking at DisplayPort 2.0 which can already barely support uncompressed 8K at 60 Hz and is basically maxed out by 5K/6K 120/144 Hz. And it's unclear if the presently introduced generation of GPUs even supports it, or if we're going to use effectively-2013-Displayport until ~2022. Note how the marketing material only talks about HDMI 2.1; DisplayPort isn't mentioned once.
  - tormeh 6y ago
    I think DisplayPort is not mentioned because Nvidia hasn't upgraded their cards to 2.0. Super disappointing. 4k/144Hz/HDR is enough for now, to be honest, but DP2 can do 84/144Hz/HDR as long as it has DSC, 244Hz with subsampling. The slow pace of development and deployment has hurt everyone for sure, but I don't think the standard is bad in itself - just a bit late.
  - hyko 6y ago
    Yeah, good points–it seems even armed with a 3090 we'll be limited to 8K/120hz/12 bit over HDMI 2.1...we'll have to wait a while before we're actually driving 8K at higher refresh rates.

TaylorAlexander 6y ago

Good thoughts all around. I will say that I am excited about 8k video for computational photography. I am studying computer vision for robotics and it is quite clear to me that a simple spherical lens with a very high resolution sensor would be a very good imaging system for a real world robot. I recently got a Panasonic GH5 and it shoots 18 megapixel “6k” 30fps video, encoded in h265 (important for higher res). I am experimenting with photogrammetry using this photo mode. The initial results are very promising. In four minutes I took a scan of a short street block in Oakland, and after processing the photogrammetry solution I have a very good dense 3D full color map of the street. The model has holes but I am slowly learning how to plan a photogrammetry capture. Currently computation takes hours but I am seeing more and more research on ways that machine learning can improve compute times and data density. See how low the visual acuity is on 5.6k spherical video here: https://youtu.be/nASvIYq3VkE https://youtu.be/nASvIYq3VkE However all this is to say that very high resolution sensors are a very good thing for robotics.

strogonoff 6y ago
Could it be faster to skip in-camera encoding and build a 3D scene based on raw scene-referred data?
- modeless 6y ago
  I think the far future of computer vision will be "event cameras" feeding directly into ML systems: https://en.wikipedia.org/wiki/Event_camera https://en.wikipedia.org/wiki/Event_camera
- namibj 6y ago
  It's not efficient to do all of this live. Skipping the H.265 parts would likely be beneficial, though.
namibj 6y ago
Hi there! So, all free/open multi-view stereo and structure-from-motion software I know of is incapable of handling rolling shutter artifacts. The problem seems to be that electronic global shutter sensors lack (some) dynamic range compared to otherwise-equal rolling shutter cameras. If you'd be interested in talking more about this, contact me/let me know (I'll get in touch if requested). My photogrammetry experiments typically encompass low-effort bulk data collection, though the search for a light-weight camera to use with a prosumer-class drone and some revelations about reconstruction quality issues inherent to older, easily-optimized algorithms for both multi-view stereo and mesh reconstruction stalled progress somewhat. In general, machine learning doesn't seem to be as much of a benefit as one might guess, when compared to applying the resources in non-ML ways to the data. Mind teasing some numbers from your street capture?

jacquesm 6y ago

That's because our operating systems treat user input as a soft real time problem when actually it should be a hard real time problem. That's why your window manager sometimes can be unresponsive. I've used a hard real time OS for a desktop for some years and the difference with consumer stuff was extreme. You get used to the computer instantly doing what you tell it to do. None of these half second or longer delays between command and result. It's like magic.

burlesona 6y ago
Which OS? What is your setup?
- deaddodo 6y ago
  Probably QNX or VxWorks.
  - neilpanchal 6y ago
    Also RTLinux kernel: https://en.wikipedia.org/wiki/RTLinux https://en.wikipedia.org/wiki/RTLinux
    - monocasa 6y ago
      RTLinux is soft real time, FWIW.
      - rudolph9 6y ago
        Can you elaborate? The wiki says it combines hard-realtime with soft-realtime. It sounds like it depends how you configure it but this is the first I’ve read of it so I could be mistaken.
        kijiki 6y ago
        They're just mistaken, RTLinux is hard realtime. It works by running a tiny real time OS on the bare metal, and then running a Linux kernel as a (lightly) paravirtualized lowest priority task. You write your hard realtime code to run on the RTOS, and then your soft/non realtime code runs on Linux. In this model, your compositor, input pipeline and (at least the UI latency sensitive part of) your applications would have to be ported to the RTOS, which makes this pretty infeasible. But it works really well if you have a some hard-realtime control loop talking to a bunch of non-realtime network IO or UI code. Would be a fun way to build a wifi controlled quadcopter. These days you probably want to use Xenomai, or possibly RTAI.

nickjj 6y ago

Fortunately a number of monitor review sites include input latency that's properly measured. For example https://www.tftcentral.co.uk/ https://www.tftcentral.co.uk/. They have a lot of popular models and super in depth reviews.

tgtweak 6y ago
Rtings pretty solid too
- zmix 6y ago
  And, should one speak German, there is https://www.prad.de/ https://www.prad.de/ too.

nomel 6y ago

> You plug your mouse into the USB ports on the monitor and it tells you how long it takes the image to change after you move the mouse. Brilliant idea. Related, "Your mouse is a terrible webcam", 2014: https://hackaday.com/2014/01/14/your-mouse-is-a-terrible-webcam/ https://hackaday.com/2014/01/14/your-mouse-is-a-terrible-web...

dan-robertson 6y ago

This does encourage optimising the OS or application part (eg double buffering and taking another frame is slow), but it doesn’t measure two significant sources of latency: The latency in the hardware before the signal makes it to the usb (for most keyboards, even specialised gaming keyboards this is like 30ms, which is 2 frames). There is also the latency from when the monitor gets a signal to when the pixels have perceptually finished transitioning. This can be 10s of ms too. So even if the OS has 0 latency, and the monitor measures that, you could still easily observe a latency of say 60ms.

modeless 6y ago
True! I just watched this video [1] they put out with more details, and it sounds like they are working with monitor and mouse manufacturers to build true end-to-end latency measurement when using supported hardware. It would be easy for manufacturers to game these numbers, but hopefully Nvidia will provide some enforcement to prevent that. Maybe they could also add a microphone to measure audio latency because that is another huge problem with modern systems. The video mentions that they get 15 ms end-to-end latency on a 360 Hz monitor in Fortnite. So that's what it takes for a modern system to finally beat the latency that you used to get on, say, an NES connected to a CRT. Still an order of magnitude above the limits of perception though [2]. [1] https://www.youtube.com/watch?v=-cXg7GQogAE https://www.youtube.com/watch?v=-cXg7GQogAE [2] https://www.youtube.com/watch?v=vOvQCPLkPt4 https://www.youtube.com/watch?v=vOvQCPLkPt4
- ksec 6y ago
  Yes! I watched that video many years ago. And then later I remember when media were hyping how "responsive" Apple Pencil was I was thinking what on earth are these people looking at. I thought to myself we would never get there because no one gives a damn about it. How the world has changed! I often wonder at what cost, could we get it down to Sub Ms. Where everything from Input, Video and Audio are sync to the precision of 1ms. There used to be very little incentive to do so. Now that Gaming has taken over I hope it will develop some market perception and value so we could finally move into that direction. The age of Instantaneous Computing.
6y ago

jayd16 6y ago

Its not really full end to end latency though. I think, at best it can measure when the the monitor thinks it received the signal from the mouse and when it perceives the new image. Latency in the monitor itself can't be measured, can it?

nightski 6y ago
Is there any reason to believe latency within the monitor itself wouldn't be relatively constant? Or would you expect large variances in monitor latency? If it was a pretty narrow distribution they could simply add on that constant factor. I'm not sure if they do this though.

ksec 6y ago

And I am surprised [1] Link hasn't pop up yet. Yes. The industry has been optimising for throughput in the past decades. It is time to bring latency back to the table. I want super low latency Computing. Street Fighter or The King of Fighters in CRT Arcade Era just felt so much more responsive. https://danluu.com/input-lag/ https://danluu.com/input-lag/

CarVac 6y ago
Using Project Slippi, you can play Super Smash Bros Melee at the same latency as on a CRT… over the internet! (Melee on a CRT has 3 frames of input lag)

amelius 6y ago

> Monitors with built-in latency measurement. They should start measuring the time it takes for monitors to recognize the signal when you press the "Source" button. Even on recent monitors, it's often 5 seconds or more.

modeless 6y ago
Yes! I would love for more review sites to benchmark this, as well as time to wake from sleep. I have a gsync monitor that has only one input port and it wakes from sleep in like 0.2 seconds. After the experience of using it for a while, if I now had to choose between gsync and fast wake from sleep I would choose fast wake from sleep every time.
SnowflakeOnIce 6y ago
My 2019 Dell monitors take 15s :|
Unklejoe 6y ago
Part of this is because of the complex equalization that must be performed when locking to the incoming signal. Basically, the device has to sweep through several different EQ settings until it achieves the combination with the best signal eye. You'd think this would be lightning fast, but as it turns out, this is often handled in firmware (with hardware support for measuring the eye). I've seen this process take upwards of two seconds. A lot of these displays probably only have one RX core, so it's not like it can remain locked to all inputs at the same time.

skohan 6y ago

In addition to input latency, with "real time" computer graphics it's basically impossible to know when your frame will actually be rendered on screen. If this information was available, it would be a lot easier to achieve smooth visuals without having to chase higher and higher frame-rates.

vlovich123 6y ago
Don’t GPU fences tell you which vsync you’re targeting? Or are you saying you want the actual estimated display time of the frame? I know the latter is definitely available for VR systems as it’s required. Having it available for 2d games probably would help marginally in cutting down the perceived latency although I don’t know if you’d be able to feel it.
- sp332 6y ago
  John Carmack has mentioned on Twitter that he spent a lot of time trying to make the frame release rate more consistent even with purpose-built VR hardware. I think game programmers found it easier to go with the gsync/freesync method of doing a vsync whenever the frame is ready instead of trying to hit a deadline.
- skohan 6y ago
  Yes I want the actual estimated display time. This would allow for more accurate interpolation.

mesh 6y ago

Here is their video on it: https://www.youtube.com/watch?v=-cXg7GQogAE https://www.youtube.com/watch?v=-cXg7GQogAE

mensetmanusman 6y ago

Don’t forget NITS. With a high NITS display, we could read outside in the bright sunlight during a pandemic.

DiabloD3 6y ago
Nits is not to be capitalized.
- mensetmanusman 6y ago
  I always yell NITS

trilinearnz 6y ago

I recall John Carmack being interested in measuring latency in exactly this way during his Occulus tenure (think he mentioned it in one of his keynotes). I'm sure he would be pleased to see this is now becoming mainstream :)

ddalex 6y ago
I did it at Intel for their Android tablets (a decade ago) using a high framerate (120Hz) webcam and counting frames. A bit cumbersome but effective.