4 ms·

Excellent writeup, thanks for posting. I was involved with efforts to achieve similar things in Windows (Direct Composition, etc.). In practice, it was enligh

Excellent writeup, thanks for posting. I was involved with efforts to achieve similar things in Windows (Direct Composition, etc.). In practice, it was enlightening to debug various apps and frameworks and discover just how long repainting can take and how many edge cases there are around slow apps, hung apps, and so on. Who knows, maybe someone there still has some of the malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.). It's definitely a hard problem space that's basically incompatible with how most GUI apps and frameworks worked from 1985-2015.

ack_complete 2y ago
IMO, the neatest trick that the DWM does is avoiding jitter/lag between the mouse cursor and window when dragging a window. Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.
- jchw 2y ago
  That trick is fun, but I wish Windows did a better job of synchronizing the visibility of the cursor sprite versus the presentation of the frames where the cursor is rendered in the frame. It's impossible for me to not notice the brief period where the cursor is missing!
  - mouse_ 2y ago
    I think it's cute. They're doing "disable hardware cursor, enable software cursor" instead of the other way around. Seeing that happen is somehow fun for me.
    - jchw 2y ago
      I think they do both simultaneously, it's just that the cursor sprite/plane is overlaid during scan out and the software cursor is rendered in the next frame to be rendered by the compositor. The cursor sprite/plane can turn on or off pretty much mid-frame, whereas the frame that's about to be rendered could be two frames away from the one that's about to be scanned out. What you actually need to do is track when the first frame with the software cursor is about to be presented, then turn off the cursor sprite/plane. I think technically speaking all of the information needed to implement this is present somewhere, but I'm sure it's quite complicated to actually implement, possibly made worse by interface layering issues :)
- flohofwoe 2y ago
  It's kinda interesting that macOS doesn't even care about this problem anymore. When moving or resizing windows on macOS the hardware mouse cursor slightly trails ahead of the window, it's not very noticeable because of the 120Hz refresh rate (meaning the framebuffer surface spends less time waiting for presentation, thus reducing rendering lag) - OTH the lag is still visible, and this sort of minor visual imperfection is quite un-Apple-like (at least for 'old Apple'). It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete (e.g. high display refresh rates making tricks to hide presentation latency obsolete). Sometimes hardware progress makes complex software solutions obsolete, but usually still with one or another tradeoff.
  - delta_p_delta_x 2y ago
    > It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete Personally, I don't think so—subpixel font rendering still has value even in the densest displays, simply because it triples the available horizontal resolution.
    - wongarsu 2y ago
      But if we continued using subpixel rendering then 1080p and 1440p displays wouldn't look as terrible, and we can't have that /s
    - account42 2y ago
      Unfortunately a low of newer displays (especially OLED ones) don't have regular horizontal RGB/BGR layouts anymore. I have seen both vertical subpixel layouts as well as triangle layouts or other funky subpixel arrangements. Even if you configure/update your software to those different subpixel layouts the result is not going to be as good.
  - mfro 2y ago
    It's pretty noticeable on older apple hardware running newer macOS. Definitely a lot of compositing performance regression in Sequoia specifically.
phkahler 2y ago
>> malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.) This is one area where Wayland should do better. It's the compositors job to resize windows and IMHO draw their decorations. The compositor has a frame to paint and it can't have to wait on applications. Unfortunately some toolkits and DEs have decided that client side decorations should still be a thing. Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).
- simcop2387 2y ago
  > Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons). https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/264 https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m... This might eventually get fixed, above is what I believe is the current proposal for handling this issue and letting apps do some amount of positioning without exposing things that a program shouldn't know about. That said this has had multiple proposals over the past 5-6 years at least and none have managed to make it all the way through. If you go through the previous ones (ext-placement and i forget the others) and ignore the angry messages involved it turns out that it's a very difficult problem to deal with in a way that isn't just a free-for-all with apps either not knowing about monitor placement, or having to handle so much detail about the displays that nothing will ever act consistently. That said, recent discussion on that latest one does look promising so maybe it'll finally happen.
  - phkahler 2y ago
    That isn't really what I was hoping for. IMHO it is wasteful for every application to remember its window placement and restore it. Not only does that require additional code in every app, but if there is some kind of policy change there is no way to make it consistent. In other words, at the compositor level one could use the previous position as a default and have an option to do something different if another window is already there. It's simply not the applications job to place itself on the desktop. There are probably use case for such self positioning functionality, but restoring to the last position is not one of them.

ack_complete 2y ago

IMO, the neatest trick that the DWM does is avoiding jitter/lag between the mouse cursor and window when dragging a window. Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.

jchw 2y ago
That trick is fun, but I wish Windows did a better job of synchronizing the visibility of the cursor sprite versus the presentation of the frames where the cursor is rendered in the frame. It's impossible for me to not notice the brief period where the cursor is missing!
- mouse_ 2y ago
  I think it's cute. They're doing "disable hardware cursor, enable software cursor" instead of the other way around. Seeing that happen is somehow fun for me.
  - jchw 2y ago
    I think they do both simultaneously, it's just that the cursor sprite/plane is overlaid during scan out and the software cursor is rendered in the next frame to be rendered by the compositor. The cursor sprite/plane can turn on or off pretty much mid-frame, whereas the frame that's about to be rendered could be two frames away from the one that's about to be scanned out. What you actually need to do is track when the first frame with the software cursor is about to be presented, then turn off the cursor sprite/plane. I think technically speaking all of the information needed to implement this is present somewhere, but I'm sure it's quite complicated to actually implement, possibly made worse by interface layering issues :)
flohofwoe 2y ago
It's kinda interesting that macOS doesn't even care about this problem anymore. When moving or resizing windows on macOS the hardware mouse cursor slightly trails ahead of the window, it's not very noticeable because of the 120Hz refresh rate (meaning the framebuffer surface spends less time waiting for presentation, thus reducing rendering lag) - OTH the lag is still visible, and this sort of minor visual imperfection is quite un-Apple-like (at least for 'old Apple'). It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete (e.g. high display refresh rates making tricks to hide presentation latency obsolete). Sometimes hardware progress makes complex software solutions obsolete, but usually still with one or another tradeoff.
- delta_p_delta_x 2y ago
  > It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete Personally, I don't think so—subpixel font rendering still has value even in the densest displays, simply because it triples the available horizontal resolution.
  - wongarsu 2y ago
    But if we continued using subpixel rendering then 1080p and 1440p displays wouldn't look as terrible, and we can't have that /s
  - account42 2y ago
    Unfortunately a low of newer displays (especially OLED ones) don't have regular horizontal RGB/BGR layouts anymore. I have seen both vertical subpixel layouts as well as triangle layouts or other funky subpixel arrangements. Even if you configure/update your software to those different subpixel layouts the result is not going to be as good.
- mfro 2y ago
  It's pretty noticeable on older apple hardware running newer macOS. Definitely a lot of compositing performance regression in Sequoia specifically.

phkahler 2y ago

>> malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.) This is one area where Wayland should do better. It's the compositors job to resize windows and IMHO draw their decorations. The compositor has a frame to paint and it can't have to wait on applications. Unfortunately some toolkits and DEs have decided that client side decorations should still be a thing. Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).

simcop2387 2y ago
> Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons). https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/264 https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m... This might eventually get fixed, above is what I believe is the current proposal for handling this issue and letting apps do some amount of positioning without exposing things that a program shouldn't know about. That said this has had multiple proposals over the past 5-6 years at least and none have managed to make it all the way through. If you go through the previous ones (ext-placement and i forget the others) and ignore the angry messages involved it turns out that it's a very difficult problem to deal with in a way that isn't just a free-for-all with apps either not knowing about monitor placement, or having to handle so much detail about the displays that nothing will ever act consistently. That said, recent discussion on that latest one does look promising so maybe it'll finally happen.
- phkahler 2y ago
  That isn't really what I was hoping for. IMHO it is wasteful for every application to remember its window placement and restore it. Not only does that require additional code in every app, but if there is some kind of policy change there is no way to make it consistent. In other words, at the compositor level one could use the previous position as a default and have an option to do something different if another window is already there. It's simply not the applications job to place itself on the desktop. There are probably use case for such self positioning functionality, but restoring to the last position is not one of them.