18 ms·

Desktop compositing latency is real

538 points by dezgeg 9y ago

antouank 9y ago
i3wm is indeed amazing. Good article. Just turned the dwm off at the win7 machine at work, and it's like a free hardware upgrade! Everything is more responsive.
- rasz 9y ago
  placebo, and as a bonus enjoy tearing videos.
wslh 9y ago
Is this something related to the latest Windows 10 updates or it was always the case? I mentioned this because, may be unrelated, there are some slow drawings in the UI after the October major update. I can notice this when I login and the desktop is drawn.
- jackmott 9y ago
  always like this for win10
- yoz-y 9y ago
  The article mainly explores the difference with DWM (Desktop Window Manager) enabled and disabled. DWM's main role is to render all of the windows into separate buffers in memory and then "compose" them on the fly. This for example avoids the effect that dragging a window over a frozen application would result in glitches. Since Windows 8 DWM can not be disabled. By default in W7 it is also enabled (the infamous Aero), but at least you can get rid of it.
- Narishma 9y ago
  It has been the case since Windows Vista. Prior to Windows 10 though, you could disable the desktop compositor.
caleblloyd 9y ago
No left margin on webpages is real too, and it annoys me
- vanderZwan 9y ago
  Are you on mobile? It centres perfectly fine for me on desktop. The linked CSS file uses this method: body { max-width: 844px; margin-left: auto; margin-right: auto; font-family: Verdana, Arial, Helvetica, sans-serif; } I guess wrapping the whole article with a div with 0.5em margin would fix it on mobile.
  - caleblloyd 9y ago
    Ah they are using margin:auto to center. I thought it must be an override since most user agents include a default body margin. Yes I'm on mobile. My OnePlus 5 hides the first one or two pixels under the bezel if looking at it straight on, so the first character on each line gets a little cutoff. Not sure if this is just my model or if other phones do this also.
    - vanderZwan 9y ago
      Either way the conclusion is the same: websites should have a minimum margin! I'm sure the author of the website is receptive to this feedback, so I sent an email. Also, Firefox (and Safari on iOS) should have "view text-optimised version" button in the URL, maybe that would help you here? I don't know if other browsers have it though.
dingo_bat 9y ago
The data speaks for itself but I'm trying to perceive any latency in my Firefox (win10) and unable to. Typing seems instantaneous to me. I hate the milliseconds the start menu takes to animate though. Win7 start was instant.
- jackmott 9y ago
  > Typing seems instantaneous to me. When the only source of latency is the extra ~16ms added by the composting, you likely can't feel it, but it adds up to other sources of latency that can make things feel bad more often. Such as: -latency added by text editors depending on their quality and the amount of work they are doing (colors, intellisense, file size, etc). -latency added by keyboards, some are worse than others -latency added by monitors. Good ones are in the 1ms range but bad ones can be as high as 40 Sadly the latency on all of these fronts has generally been trending higher as computers have gotten more powerful.
- malbertife 9y ago
  What I hate even more is when I press the Windows key and start typing an application's name into the Start menu textbox, and it misses the first two or three keypresses.
  - ben-schaaf 9y ago
    As a comparison point, gnome 3 does the right thing and buffers the input until it can handle it. On an old machine I usually have the full name typed out and enter pressed before any animation actually starts being displayed.
    - phkahler 9y ago
      How about they disable animation when windows are activated by keyboard? Or just disable it entirely ;-)
  - kevingadd 9y ago
    That's weird, I don't have that problem. What build of Win10 are you running? I wonder if it depends on the kind of keyboard you're using.
babuskov 9y ago
> Don’t you find it a bit funny that Windows 95 is actually snappier than Windows 10? Comparing them on the same hardware?
- agumonkey 9y ago
  not a problem, let's find some p2 boxes on ebay. ps: every time I boot an old desktop I always have this feeling. The new stacks are amazing, they do it all, but .. I love the immediate feeling of the old ones, even at some cost. And this comes from a compositing fetishist.
- Cthulhu_ 9y ago
  There was a similar article a while ago comparing it to an Apple II, where Apple II was snappier on its own hardware compared to a modern computer.
  - StillBored 9y ago
    I might be interested in seeing that if you can find the link. I have a transwarped IIGS, which for CPU benchmarks is slower than most of the modern emulators, but on the actual hardware its pretty amazing (particularly since it boots faster from a compaq flash card than most win10 PC's i've seen). I would guess that a USB keyboard->windows->emulator->app response->draw->Emulator->windows->gpu path is much, much longer than the IIGS keyboard poll rate draw cycle even given the 1000 cycle/sec advantage a modern PC has.
- amiga-workbench 9y ago
  Try popping the start menu open on a Windows 10 machine with a mechanical hard drive.
  - laumars 9y ago
    It wasn't much better on Windows 95 on average hardware around the time of the release of Windows 95. Heck, if you dared click [Start] as soon as the desktop displayed (read: Windows hadn't yet finished booting) then the whole OS would hang for several minutes.
ahartmetz 9y ago
I recently measured it with my phone's camera in slow motion mode. The system is an AMD Ryzen 1800X with AMD R9 280x GPU, KDE Plasma with KWin window manager in compositing mode. Key press to screen output latency was ~33 millseconds (90 fps recording, so increments of 11) in KWrite. The computer feels plenty responsive with that latency, and I hate latency... It is a full stack real world result - for comparison purposes it makes sense to measure only the software as in the article, but in reality you want to optimize everything. Especially screens can be quite bad - tens of milliseconds, up to 100 in the worst. USB lag is usually quite low - when I measured it once for low-latency serial comm it was usually < 2 ms.
- 83457 9y ago
  How are you measuring from key activation?
  - mhd 9y ago
    Yeah, that seems to be a weak point of some visual measurements, especially when laptop and other scissor keyboards score better than older switch-based ones.
  - ahartmetz 9y ago
    I don't - just filmed keyboard and screen together; key down is easy to see. I have a quality keyboard (Fujitsu KBPC PX eco) connected by PS/2, and as stated above I'd expect little extra latency from USB. As measured, anyway, there is no space for significant keyboard lag in the result. The reason why I measured latency was that I seemed to notice a change after changing GPU driver kernel options. End to end was easiest to measure. The result was close to the theoretical minimum so I stopped there.
- vetinari 9y ago
  33 ms is two frames, if your monitor is at 60 Hz. If you tried vscode or other electron app, it might be 49 ms (3 frames). These are the numbers I'm getting from 1900X with Nvidia 1080 GPU, Gnome3, 4k@60hz, but without measuring latency of the keyboard itself. Modern keyboards are another part of the problem. They can also take their sweet time since keypress until packet appears at the USB bus. See https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/
pmarreck 9y ago
100hz UWQHD monitor for work, worked for me
RyanRies 9y ago
The DWM compositor is bad for games too. The only way to take it out of the equation is to use your GPU in exclusive mode.
marcosdumay 9y ago
Why would enforcing vsync add more than 1/60s of latency to anything? This looks way more like badly designed animations than some fundamental problem coming from the hardware.
- 0xcde4c3db 9y ago
  Perhaps it ends up being multiple vsync waits for a given rendered frame? Something like the application or OpenGL driver waiting for vsync before rendering into its buffer, then the compositor waiting for the next vsync before actually compositing/flipping.
  - marcosdumay 9y ago
    I see. Locking everything until you get a vsync surely makes the code simpler. I fill that as about as likely as hardcoding the frames of animations, and displaying them all whatever device frequency you have.
  - kevingadd 9y ago
    This is a common source of delay in composited apps/games, yes. Ideally what you want is to have a completed frame ready for the compositor at least a few milliseconds before the next vertical sync arrives, but it's easy to screw that up, especially if you're getting fancy. Triple Buffering also enters the picture here (though mostly for games), because in the bad old days you had exactly two backbuffers, and if both were in use (one being scanned out to the monitor, the other your most recent completed frame) everything had to grind to a halt and wait before rendering or game code could continue. Triple buffering solved this by adding an extra frame, at the cost of an entire frame worth of display latency in exchange for your code spending less time spinning and waiting on the GPU. If someone is careless they could definitely end up with triple buffering enabled for their app (like if they're rendering using a media-oriented framework that turns it on.) The 'Fast Sync' option NVIDIA added to their drivers in the last year or two is a fix for the triple buffering problem - you get spare buffers, but instead of adding a frame of latency the GPU always grabs the most recently completed frame for scanout. Of course, if a compositor is involved you now need the compositor to do this, and then for the compositor to utilize this feature when presenting a composited desktop to the GPU. I don't think any modern compositor does this at present.
deniska 9y ago
That's why I prefer to play games in fullscreen as opposed to "borderless windowed", I have noticed quite a bit of input lag in the latter mode.
- Cthulhu_ 9y ago
  It depends on the game for me, usually I prefer borderless windowed because I tend to alt-tab out a lot.
  - jakebasile 9y ago
    Me too. It's just so much quicker to alt-tab to my browser or Telegram or iTunes, even on my beefy rig it takes multiple seconds for my desktop to take control again. In my experience the vast majority of games that I play don't suffer any perceptible ill effect from running in borderless fullscreen. If you're playing the game competitively it's another story, but for me running around shooting aliens in Destiny 2 or something I haven't noticed any degradation of my experience. There's of course the odd (usually older) game that doesn't support borderless fullscreen, but sometimes there are mods to support it.
- cybermancy 9y ago
  This is because fullscreen mode allows the use of something called "DirectDraw Exclusive mode" which bypasses Windows for making calls to the GPU. https://msdn.microsoft.com/en-us/library/windows/desktop/dd375451(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dd3...
  - nulagrithom 9y ago
    I wonder if it would be feasible to use this to reduce input lag in an editor?
    - kevingadd 9y ago
      Yes, but you lose access to any user interface components you don't paint yourself, and you can eat some lag/flickering when switching out of the app as control is returned to the compositor. If you end up needing to show the Open File picker from the OS or pop up the Print dialog you'll need to exit exclusive mode.
zokier 9y ago
I would like to see end-to-end measurements (=high speed camera footage analysis) before making final conclusions. Not saying that compositing doesn't add latency, but I feel like the system is so complex that this sort of userspace software measurement might not tell the whole story
matthew349hall 9y ago
Well his website is entirely ruined by how close the text is to the left side. It's like closer than margin:0 close.
pmoriarty 9y ago
This article is about Windows, but I wonder how Wayland on Linux measures up.
- teekert 9y ago
  Me too, since devs like those of libinput are specifically and actively addressing such latencies.
- upofadown 9y ago
  Wayland does everything with compositing. The Wayland people love v-sync because they hate tearing. So chances are that this effect applies...
  - digi_owl 9y ago
    And then Gnome goes and builds on that by hooking the mouse pointer up to the redraw...
- Sir_Cmpwn 9y ago
  It should be possible to reduce the latency quite a bit if a Wayland compositor had that goal in mind. It's something we're working on for Sway. Sometimes you have to choose beween (1) render correctly and (2) respond immediately to user feedback. When resizing windows for example, we can scale the old buffer up (stretching it) while we wait for a new buffer from the client, or we can wait to give you that feedback until the client has prepared a new buffer at the right size.
nickjj 9y ago
This is also why picking a good monitor is important for software development. Some monitors have tons of input lag (60-70ms) and that's the time it takes for you to see what you're typing to reach the display. This also includes the time it takes to see you move your mouse cursor too. I did a huge write up on picking a good monitor for development which can be found at: https://nickjanetakis.com/blog/how-to-pick-a-good-monitor-for-software-development https://nickjanetakis.com/blog/how-to-pick-a-good-monitor-fo... I ended up buying the Dell UltraSharp U2515H http://amzn.to/2jF3WHp http://amzn.to/2jF3WHp. It has very low input lag (10-15ms) and it runs natively at 2560x1440.
- perfmode 9y ago
  How is the 2715H?
  - tobyhinloopen 9y ago
    All modern (2014+) Dell IPS displays have excellent input lag
- ino 9y ago
  I have the same monitor and it's pretty good. The only thing I don't quite like is the pixel size, it's a bit small for native scaling. I kind of wish i had gone for the 27". I went with two 25" because two 27" were a bit too big for me.
  - aduitsis 9y ago
    P2715Q user here. One of the best monitors currently available, excellent color reproduction, top of the line quality. The 4K resolution however is a major problem when you are using Windows 7. Some applications will not play correctly when you try to scale up and at the end you get a mess. Windows 10 is better, but not perfect.
    - _jal 9y ago
      Ditto, have a pair of those. They are the nicest I've used in a long time. Haven't noticed the Windows issues, but then I only run a couple applications under win in a VM. No issues with Linux. They've released the UP2718Q at a serious price difference. I'd like to see one in person, but a 3x price increase requires a lot of improvement over an already great monitor.
Zardoz84 9y ago
> Don’t you find it a bit funny that Windows 95 is actually snappier than Windows 10? It’s really a shame that response times in modern computers are visibly worse than those in twenty years ago. My Amiga A1200 (Only have extra RAM) feels faster and more responsive that any modern computer with Windows or GNU/Linux desktop.
- fredley 9y ago
  Layers of abstraction take you further away from the metal. The more layers of abstraction your keypress must traverse before rendering is complete and the photons have reached your retina, the longer it will be until that happens. Layers of abstraction make complex tasks more reachable by a larger number of programmers by reducing the amount of specialist knowledge about those lower layers required to do the job. The more layers of abstraction you have, the less they have to worry about lower levels and can just get on with what they want to do.
  - wruza 9y ago
    Numerous games, including those having complex graphics and behavior, can render 120+ frames per second and realtime interactions (physics, optics, reactions) on pretty average hardware. I don’t think that game scripters who make final things like scenery or ui face complexity much harder than those in gtk/qt/wpf/htmljs widget programming. Details would be interesting though, since I’m no game developer. If true, it must be something very wrong with traditional ui systems? Edit: my apologies, I didn’t read the article first and thought it measured complete feedback like “press ctrl-f and wait for element to popup”, but I’m interested in my question regarding games anyway.
    - stult 9y ago
      It's a lot easier to optimize the APIs underlying those scripts because there are far fewer of them in even the most complicated video games than there are comparable abstractions in modern OSs. And there's more motivation. People accept the slightly lower responsiveness in normal OS interactions whereas even millisecond delays in competitive games are intolerable.
srcmap 9y ago
According to this: https://www.youtube.com/watch?v=BTURkjYJ_uk https://www.youtube.com/watch?v=BTURkjYJ_uk Firefox's servo engine can compose CSS elements/Display List together at 500 frames / second. Maybe next version of Windows / Linux desktop should use FF's servo engine?
- lukeschlather 9y ago
  That's throughput, though, not latency. I would guess that servo's input latency is significantly more than 2ms, even if their throughput for certain rendering operations is 500 fps.
- nwah1 9y ago
  Depends on the complexity of the html and CSS in question. To reproduce a full desktop environment, even just using Canvas, would be challenging and I highly doubt it would allow 500fps.
- bufferoverflow 9y ago
  Don't confuse bandwidth and latency.
- amaranth 9y ago
  Your application renders a frame then the compositor gets it and does its transformations, if any. The composited result is then rendered to the screen thus adding one frame of latency. That's why the article says one solution would be to get a 144Hz monitor, it would reduce the time between frames so an extra frame of latency wouldn't be as bad. You could potentially reduce this delay as well by having the application and the compositor in communication. Since rendering is going to be synced to vblank if you can get the application to not try to sync as well and instead just notify the compositor when it is done drawing a frame you could potentially get the application drawing and the compositor drawing in the same vblank interval. This is what Wayland and DRI3 Present let you do in the Linux world, I assume Windows has something similar but you'd need to opt-in to it so I bet nothing uses it.
- pcwalton 9y ago
  Thanks for the kind words :) But that was an artificial benchmarking mode that turned off any synchronization. It couldn't actually show the picture at hundreds of FPS, because the physical hardware can only update at 60.
baybal2 9y ago
The solution is simple - triple buffering
- bwat49 9y ago
  afaik dwm does use triple buffering
forgotmypw 9y ago
Someone recently gave me an old PowerBook G3, running Mac OS 8.6. I was amazed by how responsive the UI is compared to today's UIs, from Mac to Windows to iOS to Android. When I clicked something, it felt like there was a pushrod between the mouse button and the menu, which triggered it instantly.
- raverbashing 9y ago
  Well compositing was introduced in 10.2, I'm not sure if running Classic Mac OS was an advantage in this case
  - sandyarmstrong 9y ago
    Did you use OS X prior to 10.2? I guarantee you it was slower and worse in every way. Especially compared to classic. There is a reason they continued installing OS 9 side-by-side before 10.2.
    - galad87 9y ago
      OS X did compositing on the cpu before Quartz Extreme in 10.2
djsumdog 9y ago
Are there any such tests using i3/X11 vs sway/wayland? I'm curious about Linux input latency now.
Quarrelsome 9y ago
sorry, so they had to write some code to test what they couldn't perceive but believe they can perceive? I feel like its plausible that this is partly a psychological problem?
- mfukar 9y ago
  Well, it does say > At least I can feel the difference when typing.
  - Quarrelsome 9y ago
    I can feel people's auras. Discuss.
    - mfukar 9y ago
      I suspect you know what 'feel' and 'perceive' mean, and just being quarrelsome.
      - Quarrelsome 9y ago
        I'm suggesting its in their head and they can't actually feel or perceive it.
nwah1 9y ago
I wonder if any compositing window managers support FreeSync. That could help significantly, for any v-sync related latency. Vulkan rendering would help as well.
- floatboth 9y ago
  FreeSync only helps when you're running a GPU heavy game that can't keep up with the monitor's refresh rate (dips below 60/120/144/whatever Hz). All desktop compositors definitely can and do render at your monitor's refresh rate :) Vulkan wouldn't help much. It has less overhead (no validation in production, etc.), but GL/GLES are plenty fast for any compositing tasks. The difference might be completely negligible.
revanx_ 9y ago
The irony is that it's most often not the fault of the DWM for the latency but the applications themself. Since DWM acts as the screens double buffer, your application needs to be synchronous with the DWM frame timing, not being in sync means latency and flickering.
jheriko 9y ago
the argument about vsync and framebuffers seems mistaken. vsync only prevents partial renderers, not full renders, so disabling it does not remove latency in most cases, this article makes it sound like there is some magical way to draw directly into the buffer without it being redrawn, which is not true. the best you can get is a chance of faster drawing because you can write into the buffer is its being drawn... (and probably get some tearing) the idea that compositing is somehow slower is also very misleading... how exactly is a stacked renderer faster? i think the author is blaming a poor implementation on technical details that they only partly understand.
- badsectoracula 9y ago
  > vsync only prevents partial renderers, not full renders, so disabling it does not remove latency in most cases, V-Sync will, as the name implies, wait until the partial draw finishes. This "wait" is what is adding the latency. > the idea that compositing is somehow slower is also very misleading... how exactly is a stacked renderer faster? Yes, compositing is slower because when an application wants to draw its window needs to send the image to the compositor (or acquire the handle to the backing texture for the window that the compositor uses or whatever, that is implementation details) and then the compositor will at some point later draw it together will all the other windows (almost all compositors are V-Synced, meaning that you will see at most 60Hz updates - or whatever refresh rate your monitor is running at). This adds a very noticeable delay between the program needing to update itself and the update being visible to the user. On the other hand, without a compositor and assuming a window system based on clipping regions (like X11 and Windows without DWM) with direct to frontbuffer drawing, the application will ask the window system to prepare for drawing in the window (which usually means the window system will setup the clipping to be inside the window's visible area), then perform the drawing directly on the framebuffer and notify the window system that it is done (so that the clipping stuff can go away). Notice how nothing here waits on anything else, like v-sync (or any other interval) and how this totally ignores other windows - each window draws itself immediately when needed instead of having to orchestrate an update for all windows on the screen. Of course with the latter approach you do get tearing since you can have windows drawing themselves during a monitor refresh, but if that is a problem or not is up to the user. Personally i never care about tearing to the point that i barely notice it, yet i immediately notice any sort of V-Sync or compositor induced lag so i always try to avoid these.
igor_p 9y ago
Is there a website that demonstrates the effects of latency after pressing a key? I know there's examples of different frame rates shown with moving circles, but I don't think that's quite the same. I mean is there really a noticeable difference between say 20 and 40 ms?
tauio111 9y ago
I always thought I was the only one noticing this. With Compositing enabled, both with DWM and on GNU/Linux, the whole interaction seems to become "soft" instead of the raw that feels much nicer and snappy. From my experience it also has to do with passing through the stack to the GPU when compositing, running it all from the CPU is what makes it feel snappy. I've also been researching about removing the triple buffer vsync on W10. It seems it was possible in the first builds by replacing some system files, but that option is gone now with the recent big releases. As of that, I do not see the real reason why compositing would be needed on W10, as transparency and etc arent important factors.
- digi_owl 9y ago
  Makes me think of the "smooth scrolling" option that you can find in most web browsers. Never liked that, and first thing i hunt down after a new install. This because using it feels like scrolling through molasses for whatever reason.
fixermark 9y ago
> Actually, I don’t know why a compositing window manager should enforce V-Sync anyway? Obviously you get screen tearing without it but the option should still be there for those who want it. Every additional option (especially in the realm of video settings) opens the door for additional complexity, implementation error, and user error in unintentionally setting the undesired mode. It's perfectly understandable why window managers would settle on one or the other of two extremely different render-to-screen approaches, especially when general consensus for quite some time now in the graphics space has been that minimizing the potential for tearing is preferable.
raphlinus 9y ago
I've been experimenting with this too, in the context of the Windows front-end for xi editor. It's absolutely true that the compositor adds a frame of latency, but I have a very different take than "turn it off." First, it's possible to design an app around the compositor. Instead of sending a frame to the system, send a tree of layers. When updating the content or scrolling, just send a small delta to that tree. Further, instead of sending a single frame, send a small (~100ms) slice of time and attach animations so the motion can be silky smooth. In my experiments so far (which are not quite ready to be made public but hopefully soon), this gets you window resizing that almost exactly tracks the mouse (as opposing to lagging at least one frame behind), low latency, and excellent power usage. Further, Microsoft engineers have said that hardware overlays are coming, in which the graphics card pulls content from the windows as it's sending the video out the port, rather than taking an extra frame time to paint the windows onto a composited surface. These already exist on mobile, where power is important, but making it work for the desktop is challenging. When that happens, you get your frame back. So I think the answer is to move forward, embrace the compositor, and solve the engineering challenges, rather than move backwards, even though adding the compositor did regress latency metrics. Edit: here's the talk I was referencing that mentions overlays: https://www.youtube.com/watch?v=E3wTajGZOsA https://www.youtube.com/watch?v=E3wTajGZOsA
- pjc50 9y ago
  > Instead of sending a frame to the system, send a tree of layers. Which API does one use to do this? I'm slightly surprised that hardware overlays aren't already a feature, especially given that they're handled by the graphics card. I know there's a special API for video overlay, especially DRM video (where part of the requirement is that the system doesn't allow it near the compositor where it could be screenshotted). Can you do video "genlock" on Windows? (edit: https://msdn.microsoft.com/en-us/library/windows/desktop/dd797814(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dd7... ) I'm also wondering how VR/AR stuff handles this on Windows.
  - raphlinus 9y ago
    DirectComposition. It's been there since Windows 8, and is used by Chrome among other apps (see https://bugs.chromium.org/p/chromium/issues/detail?id=524838 https://bugs.chromium.org/p/chromium/issues/detail?id=524838). It's possible some hardware already does overlays, given that the talk I linked above was 2 years ago. I haven't researched this carefully. There's a bunch of stuff in the interface to support video and also integrated 3D content ("create swapchain for composition"), but I don't know how well it works. In my experiments and reading through the Chromium bug tracker, Microsoft's implementation of all this stuff is far from perfect, and it's hard, for example, to completely avoid artifacts on resizing.
0xJRS 9y ago
Is it really fair to compare something like Gvim to Slack though? I'm assuming Gvim is going to be magnitudes faster than an Electron app.
jd3 9y ago
further reading: https://pavelfatin.com/typing-with-pleasure/ https://pavelfatin.com/typing-with-pleasure/ https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/ In Windows 7, classic mode disabled the DWM compositor and V-Sync. It's incredibly dumb that Microsoft would arbitrarily remove that feature in Windows 10 to push their ugly as sin post-metro UI.
jsd1982 9y ago
> Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with BitBlt. This methodology alone could account for the differences in timing between Win7 and Win10. For all we know, Win10 could just be slower at getting the pixels back to the program from BitBlt, or SendInput could be slower triggering events, or a multitude of other issues. The best way to truly detect key-to-photon latency is with an external video recorder that has both the screen and keyboard in frame. Grant a few ms of noise for key travel distance.
- leeter 9y ago
  I'd be curious to see message traces and UMAPC (user mode asynchronous procedure calls) traces of this between the two. My hypothesis is that Win10 does quite a bit more in UMAPCs than win7 does in the interests of keeping the system 'responsive' even at the costs of latency. For those not aware UMAPCs only run when a thread is available in an 'alertable' state (see MSDN as that's not exactly simple to explain https://msdn.microsoft.com/en-us/library/windows/desktop/ms687069%28v=vs.85%29.aspx#alertablewaitfunctions https://msdn.microsoft.com/en-us/library/windows/desktop/ms6... ), as such they tend to wait for input or other runtime idle points unless the application makes very heavy use of windows built in asynchronous methods and alertable waits. I would also be curious to compare a D2D application versus a GDI application; as the majority of the work has gone to D2D in the last few years. Please note that D2D application in this case means one using a swap chain and device not an ID2DHWNDRenderTarget (this rasterizes and composites on the GPU but has GDI compatibility built in).
- mastax 9y ago
  As I understand it: - Compositing is done on the GPU - BitBlt is done on the CPU - Copies from GPU -> CPU are slow So, yeah, compositing adds a frame of VSync latency, but these measurements are complete bunkum.
  - leeter 9y ago
    BitBlt is done in DMA ram by the CPU, so it may be even worse than just a copy as there is likely a wait involved too to prevent shared access. Using DMA ram prohibits the GPU/Driver from doing optimizations on that ram that it could do if the buffer was in dedicated gpu-ram. This is why DX12 resources are generally always copied into non-shared buffers.
emn13 9y ago
I will gladly believe this is a real problem, but this page does not demonstrate that (at least not convincingly); the metric used is simply too poor. To quote: > I used my own hacky measurement program written in C++ that sends a keypress to the application to be benchmarked. Then it waits until the character appears on screen. Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with BitBlt. So, this is measuring some rather artificial and fairly uninteresting time. You really do need to measure the complete stack here, especially if your theory is that issues like vsync are at stake, because the level at which vsync happens can vary, and because there are interactions here that may matter. E.g. if there are 100 reasons to wait for vsync, and you remove one of them... you're still going to wait for vsync. It's not 100% clear that this measurement actually corresponds to anything real. Also, note that a compositor need not necessarily do anything on the CPU, so by trying read back the composited image, you may inadvertently be triggering some kind of unnecessary (or OS-dependent sync). E.g. it's conceivable that regardless of tearing on screen you want the read-back to work "transactionally", so you might imagine that a read requires additional synchronization that mere rendering might not. And of course: all this is really complicated, there are many moving part's we're bound to overlook. It's just common sense to try to measure something that matters, to avoid whole classes of systemic error. Ideally, you'd measure from real keypress upto visible light; but at the very least you'd want to measure from some software signal such as SendInput up to an hdmi output (...and using the same hardware and as similar as possible software). Because at the very least that captures the whole output stack, which he's interested in. Another advantage of a whole-stack measurement is that it puts things into perspective: say the additional latency is 8ms; then it's probably relevant to know at least roughly how large the latency overall is.
- astrodust 9y ago
  Once you throw in another 15+ms of keyboard and USB latency...
- chillee 9y ago
  Just another point of data with compton compositor on Arch with several text editors. https://news.ycombinator.com/item?id=14800713 https://news.ycombinator.com/item?id=14800713
mzzter 9y ago
Smartphones suffer from input latency too, though I’m unsure of the underlying cause (curious how iOS handles window/view drawing). It only seems to be getting worse, though I haven’t done tests on this. While each new model undoubtedly has better tech specs, the interface responsiveness doesn’t seem to improve.
- ahartmetz 9y ago
  Ghetto latency test: finger-scroll alternatingly up and down very quickly, see at which frequency your finger and scroll position are 180° out of phase, i.e. you finger is up while the contents are down or vice versa. Smartphones seem to be fine according to that test. Android is very good and iOS is even better.
  - mzzter 9y ago
    Hm I get about 4 up and downs per second before the scroll position is 180deg out of phase in Safari on iPhone 7+. That translates to about 125 (1000/8) ms latency?
    - ahartmetz 9y ago
      Yes, that is how it works :) That value seems surprisingly bad. My limited experience with iDevices (don't own one) has been that the position of the stationary finger on the page vs. finger on the page while scrolling (another way to measure - unless it is specifically fudged with some kind of prediction to make scrolling feel less detached) is very small. But I can't argue with data. FWIW, I like to test Android in the scroll view of the OS settings app or the address book. Those are well implemented and presumably don't add unnecessary lag.
krylon 9y ago
My work laptop (the only Windows computer I use) runs Windows 7, and I intend to keep it that way as long as Windows 7 still gets updates. This article just confirms my bias, and I freely admit I am biased. I do not like Windows very much to begin with, but as far as Windows goes, I think Windows 7 ____ing nailed it (for people without touchscreens, anyway). On a related note, I have noticed that Outlook 2013 exhibits a notable lag between a keystroke and a character appearing in the message window. I have not done any measurements, but my best guess is that it is in the order of hundreds of milliseconds. If you type fast (I like to think that I do), Outlook can keep up throughput-wise, but this lag is terribly annoying.
- mschuster91 9y ago
  > On a related note, I have noticed that Outlook 2013 exhibits a notable lag between a keystroke and a character appearing in the message window. Try switching to text-only mails, no zoom... and if you must write HTML mails, do not have an image that is larger size than the window. As soon as there is an image that doesn't fit into the window at 100% zoom Outlook begins to crawl.
  - krylon 9y ago
    My work computer is stuck with Office 2007, because I have Office 2007 Professional, and I need Access about once per year for an arcane reason. Office Pro is fairly expensive, so for the time being, I am stuck with 2007. I am still not entirely sure if I should be happy or sad about it. ;-) But I have used Outlook 2013 on coworkers' computers every now and then, and it was pretty laggy. These days, I do a lot more programming than sysadmin'ning and help desk, but when I was the IT support guy at our company, my overall impression of Office 2013 was not very good. I have seen it just stop working on a handful of computers (out of about 75-80, so to that is a lot), in such a way that I could only "fix" it by uninstalling and reinstalling Office from scratch. On one of our CAD workstations, Outlook and Autodesk Inventor started a feud where an update to MS Office caused Inventor to crash, and the subsequent reinstallation of Inventor caused Outlook to crash when we tried to write an email. (Then we reinstalled Office, and then suddenly things worked magically, so I remain clueless as to what happened.) The latter may be Autodesk's fault as much as Microsoft's (I get the vague impression that they care even less about their software crashing than Microsoft, as long as the license is paid for). But the impression I get is that MS Office has suffered quite a bit over the years. Therefore I am not entirely unhappy about being stuck on Office 2007. I do miss OneNote, the one program from their Office suite I really like, but I have org-mode, so I can manage. ;-) EDIT: Sorry for venting, that one has been building up for a long time.
mschuster91 9y ago
Well that certainly explains why a W7 with Classic theme feels faster than with the default Aero theme. Thanks to OP for digging!
h_o 9y ago
> ads in the start menu I turned off windows update quite a while ago and was considering updating my system recently until now. I'm on version 1703 Build: 15063.138 Why would I ever update now?
sravan222 9y ago
jhfkuds
sravan222 9y ago
kjsdfksd
sravan222 9y ago
Kiel Fh
rekshaw 9y ago
As the author mentions, i3wm ftw. Seriously check it out.
baxuz 9y ago
I also noticed that there's a lot more compositing latency on MacOS
- acdha 9y ago
  How did you measure this? That's contrary to what I've seen using https://itunes.apple.com/us/app/is-it-snappy/id1219667593 https://itunes.apple.com/us/app/is-it-snappy/id1219667593.
- mrmondo 9y ago
  I find the exact opposite, the UI latency of macOS seems to me a lot lower latency and more 'snappy' that Windows 7/8/10, especially with Metal2 under High Sierra.
partycoder 9y ago
If you want to have the most minimal Windows setup: - Don't use an antivirus - Stop unused services running in the background (e.g: services.msc) - Turn off all vision effects including compositing and animation Then you might want to set up a firewall to block all the nonsense like SMB, NetBIOS, etc. You can also set up a cheap old machine to act as your firewall, reverse proxy cache, antivirus/antispam, etc. You can set up a script to turn on all the printing related services when you are actually going to use a printer.
- fra0 9y ago
  Pick up an LTSB release and remove the desktop (custom shell). It looks a lot like Arch / Debian with first class hardware support.
  - partycoder 9y ago
    While LTSB looks good (fewer bundled apps, that's great), it is very dissimilar from Arch or Debian. Regarding hardware compatibility, please be aware of the fact that Debian supports many more processor architectures than Windows. Maybe Windows will support some peripherals better, but Linux has improved a lot in this respect. Chances are out of the box support for hardware is better in Linux than it is on Windows these days.
outworlder 9y ago
The compositor adds some latency, yes. It also removes latency for everything else. We no longer need to suffer when apps are slow to redraw their content whenever we move windows around.
gciruelos 9y ago
i've been using sway[0] as my wm for some time now (it's a sort of port of i3 to wayland) and it's incredible that you can actually tell that it is much faster than wms running on X. [0] http://swaywm.org/ http://swaywm.org/
- Sir_Cmpwn 9y ago
  It's funny you mention this - it's only in the past few days that we've been taking this sort of thing more seriously, and our work is unreleased!
half-kh-hacker 9y ago
> your keyboard is already slower than you might expect. An extract from the linked article:[0] > A major source of latency is key travel time. It’s not a coincidence that the quickest keyboard measured also has the shortest key travel distance by a large margin. They're not measuring from when the signal is sent from the keyboard, they're measuring from when the force begins to apply on the key. If you have a clicky or tactile switch (Cherry MX Blues, Greens, Browns, Clears, etc) then the latency measured here will be way disproportionate to how it actually /feels/. [0]: https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/
stmw 9y ago
"The big problem with latency is that it accumulates. Once some component introduces delay somewhere in the input chain you aren’t going to get it back. That’s why it’s really important to eliminate latency where you can." - a lesson that applies to many things besides the narrow case of Windows 10.
autokad 9y ago
WOW! I usually run about 7 emulators of android in one monitor, and my web browser and other stuff in the other monitor. things get really laggy sometimes, turning off this feature made HUGE difference. my experience has improved dramatically
fourthark 9y ago
I am so glad I don't notice this.
throwaway613834 9y ago
Does anybody know if DirectX and/or OpenGL also suffer from the latency? For example, would a Windows XP VM in Windows 10 have the one-frame lag?
arca_vorago 9y ago
"use an operating system that has a stacking window manager" Exactly, one more reason gnu+linux is the superior os.
oskenso 9y ago
Gnome 3 is the worst at this, especially in wayland. Moving your mouse while the compositor is busy causes input to be lost as well

538 points by dezgeg 9y ago

i3wm is indeed amazing. Good article. Just turned the dwm off at the win7 machine at work, and it's like a free hardware upgrade! Everything is more responsive.

rasz 9y ago
placebo, and as a bonus enjoy tearing videos.

wslh 9y ago

Is this something related to the latest Windows 10 updates or it was always the case? I mentioned this because, may be unrelated, there are some slow drawings in the UI after the October major update. I can notice this when I login and the desktop is drawn.

jackmott 9y ago
always like this for win10
yoz-y 9y ago
The article mainly explores the difference with DWM (Desktop Window Manager) enabled and disabled. DWM's main role is to render all of the windows into separate buffers in memory and then "compose" them on the fly. This for example avoids the effect that dragging a window over a frozen application would result in glitches. Since Windows 8 DWM can not be disabled. By default in W7 it is also enabled (the infamous Aero), but at least you can get rid of it.
Narishma 9y ago
It has been the case since Windows Vista. Prior to Windows 10 though, you could disable the desktop compositor.

caleblloyd 9y ago

No left margin on webpages is real too, and it annoys me

vanderZwan 9y ago
Are you on mobile? It centres perfectly fine for me on desktop. The linked CSS file uses this method: body { max-width: 844px; margin-left: auto; margin-right: auto; font-family: Verdana, Arial, Helvetica, sans-serif; } I guess wrapping the whole article with a div with 0.5em margin would fix it on mobile.
- caleblloyd 9y ago
  Ah they are using margin:auto to center. I thought it must be an override since most user agents include a default body margin. Yes I'm on mobile. My OnePlus 5 hides the first one or two pixels under the bezel if looking at it straight on, so the first character on each line gets a little cutoff. Not sure if this is just my model or if other phones do this also.
  - vanderZwan 9y ago
    Either way the conclusion is the same: websites should have a minimum margin! I'm sure the author of the website is receptive to this feedback, so I sent an email. Also, Firefox (and Safari on iOS) should have "view text-optimised version" button in the URL, maybe that would help you here? I don't know if other browsers have it though.

dingo_bat 9y ago

The data speaks for itself but I'm trying to perceive any latency in my Firefox (win10) and unable to. Typing seems instantaneous to me. I hate the milliseconds the start menu takes to animate though. Win7 start was instant.

jackmott 9y ago
> Typing seems instantaneous to me. When the only source of latency is the extra ~16ms added by the composting, you likely can't feel it, but it adds up to other sources of latency that can make things feel bad more often. Such as: -latency added by text editors depending on their quality and the amount of work they are doing (colors, intellisense, file size, etc). -latency added by keyboards, some are worse than others -latency added by monitors. Good ones are in the 1ms range but bad ones can be as high as 40 Sadly the latency on all of these fronts has generally been trending higher as computers have gotten more powerful.
malbertife 9y ago
What I hate even more is when I press the Windows key and start typing an application's name into the Start menu textbox, and it misses the first two or three keypresses.
- ben-schaaf 9y ago
  As a comparison point, gnome 3 does the right thing and buffers the input until it can handle it. On an old machine I usually have the full name typed out and enter pressed before any animation actually starts being displayed.
  - phkahler 9y ago
    How about they disable animation when windows are activated by keyboard? Or just disable it entirely ;-)
- kevingadd 9y ago
  That's weird, I don't have that problem. What build of Win10 are you running? I wonder if it depends on the kind of keyboard you're using.

babuskov 9y ago

> Don’t you find it a bit funny that Windows 95 is actually snappier than Windows 10? Comparing them on the same hardware?

agumonkey 9y ago
not a problem, let's find some p2 boxes on ebay. ps: every time I boot an old desktop I always have this feeling. The new stacks are amazing, they do it all, but .. I love the immediate feeling of the old ones, even at some cost. And this comes from a compositing fetishist.
Cthulhu_ 9y ago
There was a similar article a while ago comparing it to an Apple II, where Apple II was snappier on its own hardware compared to a modern computer.
- StillBored 9y ago
  I might be interested in seeing that if you can find the link. I have a transwarped IIGS, which for CPU benchmarks is slower than most of the modern emulators, but on the actual hardware its pretty amazing (particularly since it boots faster from a compaq flash card than most win10 PC's i've seen). I would guess that a USB keyboard->windows->emulator->app response->draw->Emulator->windows->gpu path is much, much longer than the IIGS keyboard poll rate draw cycle even given the 1000 cycle/sec advantage a modern PC has.
amiga-workbench 9y ago
Try popping the start menu open on a Windows 10 machine with a mechanical hard drive.
- laumars 9y ago
  It wasn't much better on Windows 95 on average hardware around the time of the release of Windows 95. Heck, if you dared click [Start] as soon as the desktop displayed (read: Windows hadn't yet finished booting) then the whole OS would hang for several minutes.

ahartmetz 9y ago

I recently measured it with my phone's camera in slow motion mode. The system is an AMD Ryzen 1800X with AMD R9 280x GPU, KDE Plasma with KWin window manager in compositing mode. Key press to screen output latency was ~33 millseconds (90 fps recording, so increments of 11) in KWrite. The computer feels plenty responsive with that latency, and I hate latency... It is a full stack real world result - for comparison purposes it makes sense to measure only the software as in the article, but in reality you want to optimize everything. Especially screens can be quite bad - tens of milliseconds, up to 100 in the worst. USB lag is usually quite low - when I measured it once for low-latency serial comm it was usually < 2 ms.

83457 9y ago
How are you measuring from key activation?
- mhd 9y ago
  Yeah, that seems to be a weak point of some visual measurements, especially when laptop and other scissor keyboards score better than older switch-based ones.
- ahartmetz 9y ago
  I don't - just filmed keyboard and screen together; key down is easy to see. I have a quality keyboard (Fujitsu KBPC PX eco) connected by PS/2, and as stated above I'd expect little extra latency from USB. As measured, anyway, there is no space for significant keyboard lag in the result. The reason why I measured latency was that I seemed to notice a change after changing GPU driver kernel options. End to end was easiest to measure. The result was close to the theoretical minimum so I stopped there.
vetinari 9y ago
33 ms is two frames, if your monitor is at 60 Hz. If you tried vscode or other electron app, it might be 49 ms (3 frames). These are the numbers I'm getting from 1900X with Nvidia 1080 GPU, Gnome3, 4k@60hz, but without measuring latency of the keyboard itself. Modern keyboards are another part of the problem. They can also take their sweet time since keypress until packet appears at the USB bus. See https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/

pmarreck 9y ago

100hz UWQHD monitor for work, worked for me

RyanRies 9y ago

The DWM compositor is bad for games too. The only way to take it out of the equation is to use your GPU in exclusive mode.

marcosdumay 9y ago

Why would enforcing vsync add more than 1/60s of latency to anything? This looks way more like badly designed animations than some fundamental problem coming from the hardware.

0xcde4c3db 9y ago
Perhaps it ends up being multiple vsync waits for a given rendered frame? Something like the application or OpenGL driver waiting for vsync before rendering into its buffer, then the compositor waiting for the next vsync before actually compositing/flipping.
- marcosdumay 9y ago
  I see. Locking everything until you get a vsync surely makes the code simpler. I fill that as about as likely as hardcoding the frames of animations, and displaying them all whatever device frequency you have.
- kevingadd 9y ago
  This is a common source of delay in composited apps/games, yes. Ideally what you want is to have a completed frame ready for the compositor at least a few milliseconds before the next vertical sync arrives, but it's easy to screw that up, especially if you're getting fancy. Triple Buffering also enters the picture here (though mostly for games), because in the bad old days you had exactly two backbuffers, and if both were in use (one being scanned out to the monitor, the other your most recent completed frame) everything had to grind to a halt and wait before rendering or game code could continue. Triple buffering solved this by adding an extra frame, at the cost of an entire frame worth of display latency in exchange for your code spending less time spinning and waiting on the GPU. If someone is careless they could definitely end up with triple buffering enabled for their app (like if they're rendering using a media-oriented framework that turns it on.) The 'Fast Sync' option NVIDIA added to their drivers in the last year or two is a fix for the triple buffering problem - you get spare buffers, but instead of adding a frame of latency the GPU always grabs the most recently completed frame for scanout. Of course, if a compositor is involved you now need the compositor to do this, and then for the compositor to utilize this feature when presenting a composited desktop to the GPU. I don't think any modern compositor does this at present.

deniska 9y ago

That's why I prefer to play games in fullscreen as opposed to "borderless windowed", I have noticed quite a bit of input lag in the latter mode.

Cthulhu_ 9y ago
It depends on the game for me, usually I prefer borderless windowed because I tend to alt-tab out a lot.
- jakebasile 9y ago
  Me too. It's just so much quicker to alt-tab to my browser or Telegram or iTunes, even on my beefy rig it takes multiple seconds for my desktop to take control again. In my experience the vast majority of games that I play don't suffer any perceptible ill effect from running in borderless fullscreen. If you're playing the game competitively it's another story, but for me running around shooting aliens in Destiny 2 or something I haven't noticed any degradation of my experience. There's of course the odd (usually older) game that doesn't support borderless fullscreen, but sometimes there are mods to support it.
cybermancy 9y ago
This is because fullscreen mode allows the use of something called "DirectDraw Exclusive mode" which bypasses Windows for making calls to the GPU. https://msdn.microsoft.com/en-us/library/windows/desktop/dd375451(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dd3...
- nulagrithom 9y ago
  I wonder if it would be feasible to use this to reduce input lag in an editor?
  - kevingadd 9y ago
    Yes, but you lose access to any user interface components you don't paint yourself, and you can eat some lag/flickering when switching out of the app as control is returned to the compositor. If you end up needing to show the Open File picker from the OS or pop up the Print dialog you'll need to exit exclusive mode.

zokier 9y ago

I would like to see end-to-end measurements (=high speed camera footage analysis) before making final conclusions. Not saying that compositing doesn't add latency, but I feel like the system is so complex that this sort of userspace software measurement might not tell the whole story

matthew349hall 9y ago

Well his website is entirely ruined by how close the text is to the left side. It's like closer than margin:0 close.

pmoriarty 9y ago

This article is about Windows, but I wonder how Wayland on Linux measures up.

teekert 9y ago
Me too, since devs like those of libinput are specifically and actively addressing such latencies.
upofadown 9y ago
Wayland does everything with compositing. The Wayland people love v-sync because they hate tearing. So chances are that this effect applies...
- digi_owl 9y ago
  And then Gnome goes and builds on that by hooking the mouse pointer up to the redraw...
Sir_Cmpwn 9y ago
It should be possible to reduce the latency quite a bit if a Wayland compositor had that goal in mind. It's something we're working on for Sway. Sometimes you have to choose beween (1) render correctly and (2) respond immediately to user feedback. When resizing windows for example, we can scale the old buffer up (stretching it) while we wait for a new buffer from the client, or we can wait to give you that feedback until the client has prepared a new buffer at the right size.

nickjj 9y ago

This is also why picking a good monitor is important for software development. Some monitors have tons of input lag (60-70ms) and that's the time it takes for you to see what you're typing to reach the display. This also includes the time it takes to see you move your mouse cursor too. I did a huge write up on picking a good monitor for development which can be found at: https://nickjanetakis.com/blog/how-to-pick-a-good-monitor-for-software-development https://nickjanetakis.com/blog/how-to-pick-a-good-monitor-fo... I ended up buying the Dell UltraSharp U2515H http://amzn.to/2jF3WHp http://amzn.to/2jF3WHp. It has very low input lag (10-15ms) and it runs natively at 2560x1440.

perfmode 9y ago
How is the 2715H?
- tobyhinloopen 9y ago
  All modern (2014+) Dell IPS displays have excellent input lag
ino 9y ago
I have the same monitor and it's pretty good. The only thing I don't quite like is the pixel size, it's a bit small for native scaling. I kind of wish i had gone for the 27". I went with two 25" because two 27" were a bit too big for me.
- aduitsis 9y ago
  P2715Q user here. One of the best monitors currently available, excellent color reproduction, top of the line quality. The 4K resolution however is a major problem when you are using Windows 7. Some applications will not play correctly when you try to scale up and at the end you get a mess. Windows 10 is better, but not perfect.
  - _jal 9y ago
    Ditto, have a pair of those. They are the nicest I've used in a long time. Haven't noticed the Windows issues, but then I only run a couple applications under win in a VM. No issues with Linux. They've released the UP2718Q at a serious price difference. I'd like to see one in person, but a 3x price increase requires a lot of improvement over an already great monitor.

Zardoz84 9y ago

> Don’t you find it a bit funny that Windows 95 is actually snappier than Windows 10? It’s really a shame that response times in modern computers are visibly worse than those in twenty years ago. My Amiga A1200 (Only have extra RAM) feels faster and more responsive that any modern computer with Windows or GNU/Linux desktop.

fredley 9y ago
Layers of abstraction take you further away from the metal. The more layers of abstraction your keypress must traverse before rendering is complete and the photons have reached your retina, the longer it will be until that happens. Layers of abstraction make complex tasks more reachable by a larger number of programmers by reducing the amount of specialist knowledge about those lower layers required to do the job. The more layers of abstraction you have, the less they have to worry about lower levels and can just get on with what they want to do.
- wruza 9y ago
  Numerous games, including those having complex graphics and behavior, can render 120+ frames per second and realtime interactions (physics, optics, reactions) on pretty average hardware. I don’t think that game scripters who make final things like scenery or ui face complexity much harder than those in gtk/qt/wpf/htmljs widget programming. Details would be interesting though, since I’m no game developer. If true, it must be something very wrong with traditional ui systems? Edit: my apologies, I didn’t read the article first and thought it measured complete feedback like “press ctrl-f and wait for element to popup”, but I’m interested in my question regarding games anyway.
  - stult 9y ago
    It's a lot easier to optimize the APIs underlying those scripts because there are far fewer of them in even the most complicated video games than there are comparable abstractions in modern OSs. And there's more motivation. People accept the slightly lower responsiveness in normal OS interactions whereas even millisecond delays in competitive games are intolerable.

srcmap 9y ago

According to this: https://www.youtube.com/watch?v=BTURkjYJ_uk https://www.youtube.com/watch?v=BTURkjYJ_uk Firefox's servo engine can compose CSS elements/Display List together at 500 frames / second. Maybe next version of Windows / Linux desktop should use FF's servo engine?

lukeschlather 9y ago
That's throughput, though, not latency. I would guess that servo's input latency is significantly more than 2ms, even if their throughput for certain rendering operations is 500 fps.
nwah1 9y ago
Depends on the complexity of the html and CSS in question. To reproduce a full desktop environment, even just using Canvas, would be challenging and I highly doubt it would allow 500fps.
bufferoverflow 9y ago
Don't confuse bandwidth and latency.
amaranth 9y ago
Your application renders a frame then the compositor gets it and does its transformations, if any. The composited result is then rendered to the screen thus adding one frame of latency. That's why the article says one solution would be to get a 144Hz monitor, it would reduce the time between frames so an extra frame of latency wouldn't be as bad. You could potentially reduce this delay as well by having the application and the compositor in communication. Since rendering is going to be synced to vblank if you can get the application to not try to sync as well and instead just notify the compositor when it is done drawing a frame you could potentially get the application drawing and the compositor drawing in the same vblank interval. This is what Wayland and DRI3 Present let you do in the Linux world, I assume Windows has something similar but you'd need to opt-in to it so I bet nothing uses it.
pcwalton 9y ago
Thanks for the kind words :) But that was an artificial benchmarking mode that turned off any synchronization. It couldn't actually show the picture at hundreds of FPS, because the physical hardware can only update at 60.

baybal2 9y ago

The solution is simple - triple buffering

bwat49 9y ago
afaik dwm does use triple buffering

forgotmypw 9y ago

Someone recently gave me an old PowerBook G3, running Mac OS 8.6. I was amazed by how responsive the UI is compared to today's UIs, from Mac to Windows to iOS to Android. When I clicked something, it felt like there was a pushrod between the mouse button and the menu, which triggered it instantly.

raverbashing 9y ago
Well compositing was introduced in 10.2, I'm not sure if running Classic Mac OS was an advantage in this case
- sandyarmstrong 9y ago
  Did you use OS X prior to 10.2? I guarantee you it was slower and worse in every way. Especially compared to classic. There is a reason they continued installing OS 9 side-by-side before 10.2.
  - galad87 9y ago
    OS X did compositing on the cpu before Quartz Extreme in 10.2

djsumdog 9y ago

Are there any such tests using i3/X11 vs sway/wayland? I'm curious about Linux input latency now.

Quarrelsome 9y ago

sorry, so they had to write some code to test what they couldn't perceive but believe they can perceive? I feel like its plausible that this is partly a psychological problem?

mfukar 9y ago
Well, it does say > At least I can feel the difference when typing.
- Quarrelsome 9y ago
  I can feel people's auras. Discuss.
  - mfukar 9y ago
    I suspect you know what 'feel' and 'perceive' mean, and just being quarrelsome.
    - Quarrelsome 9y ago
      I'm suggesting its in their head and they can't actually feel or perceive it.

nwah1 9y ago

I wonder if any compositing window managers support FreeSync. That could help significantly, for any v-sync related latency. Vulkan rendering would help as well.

floatboth 9y ago
FreeSync only helps when you're running a GPU heavy game that can't keep up with the monitor's refresh rate (dips below 60/120/144/whatever Hz). All desktop compositors definitely can and do render at your monitor's refresh rate :) Vulkan wouldn't help much. It has less overhead (no validation in production, etc.), but GL/GLES are plenty fast for any compositing tasks. The difference might be completely negligible.

revanx_ 9y ago

The irony is that it's most often not the fault of the DWM for the latency but the applications themself. Since DWM acts as the screens double buffer, your application needs to be synchronous with the DWM frame timing, not being in sync means latency and flickering.

jheriko 9y ago

the argument about vsync and framebuffers seems mistaken. vsync only prevents partial renderers, not full renders, so disabling it does not remove latency in most cases, this article makes it sound like there is some magical way to draw directly into the buffer without it being redrawn, which is not true. the best you can get is a chance of faster drawing because you can write into the buffer is its being drawn... (and probably get some tearing) the idea that compositing is somehow slower is also very misleading... how exactly is a stacked renderer faster? i think the author is blaming a poor implementation on technical details that they only partly understand.

badsectoracula 9y ago
> vsync only prevents partial renderers, not full renders, so disabling it does not remove latency in most cases, V-Sync will, as the name implies, wait until the partial draw finishes. This "wait" is what is adding the latency. > the idea that compositing is somehow slower is also very misleading... how exactly is a stacked renderer faster? Yes, compositing is slower because when an application wants to draw its window needs to send the image to the compositor (or acquire the handle to the backing texture for the window that the compositor uses or whatever, that is implementation details) and then the compositor will at some point later draw it together will all the other windows (almost all compositors are V-Synced, meaning that you will see at most 60Hz updates - or whatever refresh rate your monitor is running at). This adds a very noticeable delay between the program needing to update itself and the update being visible to the user. On the other hand, without a compositor and assuming a window system based on clipping regions (like X11 and Windows without DWM) with direct to frontbuffer drawing, the application will ask the window system to prepare for drawing in the window (which usually means the window system will setup the clipping to be inside the window's visible area), then perform the drawing directly on the framebuffer and notify the window system that it is done (so that the clipping stuff can go away). Notice how nothing here waits on anything else, like v-sync (or any other interval) and how this totally ignores other windows - each window draws itself immediately when needed instead of having to orchestrate an update for all windows on the screen. Of course with the latter approach you do get tearing since you can have windows drawing themselves during a monitor refresh, but if that is a problem or not is up to the user. Personally i never care about tearing to the point that i barely notice it, yet i immediately notice any sort of V-Sync or compositor induced lag so i always try to avoid these.

igor_p 9y ago

Is there a website that demonstrates the effects of latency after pressing a key? I know there's examples of different frame rates shown with moving circles, but I don't think that's quite the same. I mean is there really a noticeable difference between say 20 and 40 ms?

tauio111 9y ago

I always thought I was the only one noticing this. With Compositing enabled, both with DWM and on GNU/Linux, the whole interaction seems to become "soft" instead of the raw that feels much nicer and snappy. From my experience it also has to do with passing through the stack to the GPU when compositing, running it all from the CPU is what makes it feel snappy. I've also been researching about removing the triple buffer vsync on W10. It seems it was possible in the first builds by replacing some system files, but that option is gone now with the recent big releases. As of that, I do not see the real reason why compositing would be needed on W10, as transparency and etc arent important factors.

digi_owl 9y ago
Makes me think of the "smooth scrolling" option that you can find in most web browsers. Never liked that, and first thing i hunt down after a new install. This because using it feels like scrolling through molasses for whatever reason.

fixermark 9y ago

> Actually, I don’t know why a compositing window manager should enforce V-Sync anyway? Obviously you get screen tearing without it but the option should still be there for those who want it. Every additional option (especially in the realm of video settings) opens the door for additional complexity, implementation error, and user error in unintentionally setting the undesired mode. It's perfectly understandable why window managers would settle on one or the other of two extremely different render-to-screen approaches, especially when general consensus for quite some time now in the graphics space has been that minimizing the potential for tearing is preferable.

raphlinus 9y ago

I've been experimenting with this too, in the context of the Windows front-end for xi editor. It's absolutely true that the compositor adds a frame of latency, but I have a very different take than "turn it off." First, it's possible to design an app around the compositor. Instead of sending a frame to the system, send a tree of layers. When updating the content or scrolling, just send a small delta to that tree. Further, instead of sending a single frame, send a small (~100ms) slice of time and attach animations so the motion can be silky smooth. In my experiments so far (which are not quite ready to be made public but hopefully soon), this gets you window resizing that almost exactly tracks the mouse (as opposing to lagging at least one frame behind), low latency, and excellent power usage. Further, Microsoft engineers have said that hardware overlays are coming, in which the graphics card pulls content from the windows as it's sending the video out the port, rather than taking an extra frame time to paint the windows onto a composited surface. These already exist on mobile, where power is important, but making it work for the desktop is challenging. When that happens, you get your frame back. So I think the answer is to move forward, embrace the compositor, and solve the engineering challenges, rather than move backwards, even though adding the compositor did regress latency metrics. Edit: here's the talk I was referencing that mentions overlays: https://www.youtube.com/watch?v=E3wTajGZOsA https://www.youtube.com/watch?v=E3wTajGZOsA

pjc50 9y ago
> Instead of sending a frame to the system, send a tree of layers. Which API does one use to do this? I'm slightly surprised that hardware overlays aren't already a feature, especially given that they're handled by the graphics card. I know there's a special API for video overlay, especially DRM video (where part of the requirement is that the system doesn't allow it near the compositor where it could be screenshotted). Can you do video "genlock" on Windows? (edit: https://msdn.microsoft.com/en-us/library/windows/desktop/dd797814(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/dd7... ) I'm also wondering how VR/AR stuff handles this on Windows.
- raphlinus 9y ago
  DirectComposition. It's been there since Windows 8, and is used by Chrome among other apps (see https://bugs.chromium.org/p/chromium/issues/detail?id=524838 https://bugs.chromium.org/p/chromium/issues/detail?id=524838). It's possible some hardware already does overlays, given that the talk I linked above was 2 years ago. I haven't researched this carefully. There's a bunch of stuff in the interface to support video and also integrated 3D content ("create swapchain for composition"), but I don't know how well it works. In my experiments and reading through the Chromium bug tracker, Microsoft's implementation of all this stuff is far from perfect, and it's hard, for example, to completely avoid artifacts on resizing.

0xJRS 9y ago

Is it really fair to compare something like Gvim to Slack though? I'm assuming Gvim is going to be magnitudes faster than an Electron app.

jd3 9y ago

further reading: https://pavelfatin.com/typing-with-pleasure/ https://pavelfatin.com/typing-with-pleasure/ https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/ In Windows 7, classic mode disabled the DWM compositor and V-Sync. It's incredibly dumb that Microsoft would arbitrarily remove that feature in Windows 10 to push their ugly as sin post-metro UI.

jsd1982 9y ago

> Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with BitBlt. This methodology alone could account for the differences in timing between Win7 and Win10. For all we know, Win10 could just be slower at getting the pixels back to the program from BitBlt, or SendInput could be slower triggering events, or a multitude of other issues. The best way to truly detect key-to-photon latency is with an external video recorder that has both the screen and keyboard in frame. Grant a few ms of noise for key travel distance.

leeter 9y ago
I'd be curious to see message traces and UMAPC (user mode asynchronous procedure calls) traces of this between the two. My hypothesis is that Win10 does quite a bit more in UMAPCs than win7 does in the interests of keeping the system 'responsive' even at the costs of latency. For those not aware UMAPCs only run when a thread is available in an 'alertable' state (see MSDN as that's not exactly simple to explain https://msdn.microsoft.com/en-us/library/windows/desktop/ms687069%28v=vs.85%29.aspx#alertablewaitfunctions https://msdn.microsoft.com/en-us/library/windows/desktop/ms6... ), as such they tend to wait for input or other runtime idle points unless the application makes very heavy use of windows built in asynchronous methods and alertable waits. I would also be curious to compare a D2D application versus a GDI application; as the majority of the work has gone to D2D in the last few years. Please note that D2D application in this case means one using a swap chain and device not an ID2DHWNDRenderTarget (this rasterizes and composites on the GPU but has GDI compatibility built in).
mastax 9y ago
As I understand it: - Compositing is done on the GPU - BitBlt is done on the CPU - Copies from GPU -> CPU are slow So, yeah, compositing adds a frame of VSync latency, but these measurements are complete bunkum.
- leeter 9y ago
  BitBlt is done in DMA ram by the CPU, so it may be even worse than just a copy as there is likely a wait involved too to prevent shared access. Using DMA ram prohibits the GPU/Driver from doing optimizations on that ram that it could do if the buffer was in dedicated gpu-ram. This is why DX12 resources are generally always copied into non-shared buffers.

emn13 9y ago

I will gladly believe this is a real problem, but this page does not demonstrate that (at least not convincingly); the metric used is simply too poor. To quote: > I used my own hacky measurement program written in C++ that sends a keypress to the application to be benchmarked. Then it waits until the character appears on screen. Virtual keypresses were sent with WinAPIs SendInput and pixels copied off screen with BitBlt. So, this is measuring some rather artificial and fairly uninteresting time. You really do need to measure the complete stack here, especially if your theory is that issues like vsync are at stake, because the level at which vsync happens can vary, and because there are interactions here that may matter. E.g. if there are 100 reasons to wait for vsync, and you remove one of them... you're still going to wait for vsync. It's not 100% clear that this measurement actually corresponds to anything real. Also, note that a compositor need not necessarily do anything on the CPU, so by trying read back the composited image, you may inadvertently be triggering some kind of unnecessary (or OS-dependent sync). E.g. it's conceivable that regardless of tearing on screen you want the read-back to work "transactionally", so you might imagine that a read requires additional synchronization that mere rendering might not. And of course: all this is really complicated, there are many moving part's we're bound to overlook. It's just common sense to try to measure something that matters, to avoid whole classes of systemic error. Ideally, you'd measure from real keypress upto visible light; but at the very least you'd want to measure from some software signal such as SendInput up to an hdmi output (...and using the same hardware and as similar as possible software). Because at the very least that captures the whole output stack, which he's interested in. Another advantage of a whole-stack measurement is that it puts things into perspective: say the additional latency is 8ms; then it's probably relevant to know at least roughly how large the latency overall is.

astrodust 9y ago
Once you throw in another 15+ms of keyboard and USB latency...
chillee 9y ago
Just another point of data with compton compositor on Arch with several text editors. https://news.ycombinator.com/item?id=14800713 https://news.ycombinator.com/item?id=14800713

mzzter 9y ago

Smartphones suffer from input latency too, though I’m unsure of the underlying cause (curious how iOS handles window/view drawing). It only seems to be getting worse, though I haven’t done tests on this. While each new model undoubtedly has better tech specs, the interface responsiveness doesn’t seem to improve.

ahartmetz 9y ago
Ghetto latency test: finger-scroll alternatingly up and down very quickly, see at which frequency your finger and scroll position are 180° out of phase, i.e. you finger is up while the contents are down or vice versa. Smartphones seem to be fine according to that test. Android is very good and iOS is even better.
- mzzter 9y ago
  Hm I get about 4 up and downs per second before the scroll position is 180deg out of phase in Safari on iPhone 7+. That translates to about 125 (1000/8) ms latency?
  - ahartmetz 9y ago
    Yes, that is how it works :) That value seems surprisingly bad. My limited experience with iDevices (don't own one) has been that the position of the stationary finger on the page vs. finger on the page while scrolling (another way to measure - unless it is specifically fudged with some kind of prediction to make scrolling feel less detached) is very small. But I can't argue with data. FWIW, I like to test Android in the scroll view of the OS settings app or the address book. Those are well implemented and presumably don't add unnecessary lag.

krylon 9y ago

My work laptop (the only Windows computer I use) runs Windows 7, and I intend to keep it that way as long as Windows 7 still gets updates. This article just confirms my bias, and I freely admit I am biased. I do not like Windows very much to begin with, but as far as Windows goes, I think Windows 7 ____ing nailed it (for people without touchscreens, anyway). On a related note, I have noticed that Outlook 2013 exhibits a notable lag between a keystroke and a character appearing in the message window. I have not done any measurements, but my best guess is that it is in the order of hundreds of milliseconds. If you type fast (I like to think that I do), Outlook can keep up throughput-wise, but this lag is terribly annoying.

mschuster91 9y ago
> On a related note, I have noticed that Outlook 2013 exhibits a notable lag between a keystroke and a character appearing in the message window. Try switching to text-only mails, no zoom... and if you must write HTML mails, do not have an image that is larger size than the window. As soon as there is an image that doesn't fit into the window at 100% zoom Outlook begins to crawl.
- krylon 9y ago
  My work computer is stuck with Office 2007, because I have Office 2007 Professional, and I need Access about once per year for an arcane reason. Office Pro is fairly expensive, so for the time being, I am stuck with 2007. I am still not entirely sure if I should be happy or sad about it. ;-) But I have used Outlook 2013 on coworkers' computers every now and then, and it was pretty laggy. These days, I do a lot more programming than sysadmin'ning and help desk, but when I was the IT support guy at our company, my overall impression of Office 2013 was not very good. I have seen it just stop working on a handful of computers (out of about 75-80, so to that is a lot), in such a way that I could only "fix" it by uninstalling and reinstalling Office from scratch. On one of our CAD workstations, Outlook and Autodesk Inventor started a feud where an update to MS Office caused Inventor to crash, and the subsequent reinstallation of Inventor caused Outlook to crash when we tried to write an email. (Then we reinstalled Office, and then suddenly things worked magically, so I remain clueless as to what happened.) The latter may be Autodesk's fault as much as Microsoft's (I get the vague impression that they care even less about their software crashing than Microsoft, as long as the license is paid for). But the impression I get is that MS Office has suffered quite a bit over the years. Therefore I am not entirely unhappy about being stuck on Office 2007. I do miss OneNote, the one program from their Office suite I really like, but I have org-mode, so I can manage. ;-) EDIT: Sorry for venting, that one has been building up for a long time.

mschuster91 9y ago

Well that certainly explains why a W7 with Classic theme feels faster than with the default Aero theme. Thanks to OP for digging!

h_o 9y ago

> ads in the start menu I turned off windows update quite a while ago and was considering updating my system recently until now. I'm on version 1703 Build: 15063.138 Why would I ever update now?

sravan222 9y ago

jhfkuds

sravan222 9y ago

kjsdfksd

sravan222 9y ago

Kiel Fh

rekshaw 9y ago

As the author mentions, i3wm ftw. Seriously check it out.

baxuz 9y ago

I also noticed that there's a lot more compositing latency on MacOS

acdha 9y ago
How did you measure this? That's contrary to what I've seen using https://itunes.apple.com/us/app/is-it-snappy/id1219667593 https://itunes.apple.com/us/app/is-it-snappy/id1219667593.
mrmondo 9y ago
I find the exact opposite, the UI latency of macOS seems to me a lot lower latency and more 'snappy' that Windows 7/8/10, especially with Metal2 under High Sierra.

partycoder 9y ago

If you want to have the most minimal Windows setup: - Don't use an antivirus - Stop unused services running in the background (e.g: services.msc) - Turn off all vision effects including compositing and animation Then you might want to set up a firewall to block all the nonsense like SMB, NetBIOS, etc. You can also set up a cheap old machine to act as your firewall, reverse proxy cache, antivirus/antispam, etc. You can set up a script to turn on all the printing related services when you are actually going to use a printer.

fra0 9y ago
Pick up an LTSB release and remove the desktop (custom shell). It looks a lot like Arch / Debian with first class hardware support.
- partycoder 9y ago
  While LTSB looks good (fewer bundled apps, that's great), it is very dissimilar from Arch or Debian. Regarding hardware compatibility, please be aware of the fact that Debian supports many more processor architectures than Windows. Maybe Windows will support some peripherals better, but Linux has improved a lot in this respect. Chances are out of the box support for hardware is better in Linux than it is on Windows these days.

outworlder 9y ago

The compositor adds some latency, yes. It also removes latency for everything else. We no longer need to suffer when apps are slow to redraw their content whenever we move windows around.

gciruelos 9y ago

i've been using sway[0] as my wm for some time now (it's a sort of port of i3 to wayland) and it's incredible that you can actually tell that it is much faster than wms running on X. [0] http://swaywm.org/ http://swaywm.org/

Sir_Cmpwn 9y ago
It's funny you mention this - it's only in the past few days that we've been taking this sort of thing more seriously, and our work is unreleased!

half-kh-hacker 9y ago

> your keyboard is already slower than you might expect. An extract from the linked article:[0] > A major source of latency is key travel time. It’s not a coincidence that the quickest keyboard measured also has the shortest key travel distance by a large margin. They're not measuring from when the signal is sent from the keyboard, they're measuring from when the force begins to apply on the key. If you have a clicky or tactile switch (Cherry MX Blues, Greens, Browns, Clears, etc) then the latency measured here will be way disproportionate to how it actually /feels/. [0]: https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/

stmw 9y ago

"The big problem with latency is that it accumulates. Once some component introduces delay somewhere in the input chain you aren’t going to get it back. That’s why it’s really important to eliminate latency where you can." - a lesson that applies to many things besides the narrow case of Windows 10.

autokad 9y ago

WOW! I usually run about 7 emulators of android in one monitor, and my web browser and other stuff in the other monitor. things get really laggy sometimes, turning off this feature made HUGE difference. my experience has improved dramatically

fourthark 9y ago

I am so glad I don't notice this.

throwaway613834 9y ago

Does anybody know if DirectX and/or OpenGL also suffer from the latency? For example, would a Windows XP VM in Windows 10 have the one-frame lag?

arca_vorago 9y ago

"use an operating system that has a stacking window manager" Exactly, one more reason gnu+linux is the superior os.

oskenso 9y ago

Gnome 3 is the worst at this, especially in wayland. Moving your mouse while the compositor is busy causes input to be lost as well