4 ms·

The most viable approach these days is for 2D is to use the 3D hardware. There's no standard, usable API for 2D accelerated drawing the way there is for 3D, nor

The most viable approach these days is for 2D is to use the 3D hardware. There's no standard, usable API for 2D accelerated drawing the way there is for 3D, nor does it quite make sense for there to be one. (No, OpenVG is not viable. No, Xrender is not viable. cairo and Skia both use the 3D hardware in combination with a CPU render engine.)

pcwalton 3y ago
For the most part that's true, but simple 2D compositing is a bit of a different beast, because it can sometimes be done at scanout time, saving a blit. Last I checked, (non-Android) Linux rarely makes use of this except for the mouse cursor. But in general you can save a good bit of energy and memory bandwidth on HiDPI displays if you try to use 2D hardware layers where you can. You can virtually never use them for the UI itself, because they're far too limited, but the windowing system can often use them to composite windows together. It'd be nice if Wayland compositors made more use of this, e.g. to avoid having to blit the foreground window every frame.
- kimixa 3y ago
  I used to work on mobile graphics and the android HWC stack. The scanout-time hardware was often less useful that you might think - only in dynamic scenes where the GPU is otherwise idle (like playing video possibly with a static UI overlay was the premier use case). For static scenes it's more efficient to render out to a buffer (using the GPU as the scanout overlay pipes often had limited feedback capability) and just output that using overlays disabled. It didn't take many frames for that to be worth it. For apps that were animating or otherwise updating it's window, most UI toolkits used the GPU for widget rendering. And often the scanout pipes didn't hook into the (relatively large) system caches like the GPU did, so there were times it was again faster to composite the screen on the GPU to a single scanout buffer than flush already cached data, the get the scanout hardware to read it back from the memory bus. And there weren't as cheap as people thought - one stat I remember was that the total area of the GPU on the omap4 platform was smaller than the display pipes. Though that is now a pretty old chip, and always had a bit of focus on "multimedia".
  - deleted 3y ago
    [deleted]
  - kllrnohj 3y ago
    I think your information is quite outdated. The HWC overlay planes are heavily used, you can see this trivially just doing a 'dumpsys SurfaceFlinger' or grabbing a systrace/perfetto trace. When it falls back to GPU composition it's very obvious as there's a significant hit to latency and more GPU contention. The overlay capabilities of the modern Snapdragons are also quite absurd. They support like upwards of a dozen overlays now and even have FP16 extended sRGB support. Some HWCs (like the one in the steam deck) even have per plane 3D LUTs for HDR tone mapping (ex https://github.com/ValveSoftware/gamescope/blob/master/src/docs/Steam%20Deck%20Display%20Pipeline.png https://github.com/ValveSoftware/gamescope/blob/master/src/d... ) The composition is bandwidth heavy of course, but for static scenes there's a cache after the HWC in the form of panel self refresh.
- Jasper_ 3y ago
  CRTC planes and scanout-time compositing makes sense, and Wayland compositors do use them, even for non-cursor surfaces. It's simply not something an application can use general-purpose and guarantee (though see the recent GtkSurfaceOffload stuff for the latest attempt at it). Personally, I don't see it as a "2D drawing API", it doesn't accelerate anything special about 2D, only blits and transforms, which a 3D API will eat for breakfast.
- Zardoz84 3y ago
  what happened with these VESA "2d accelerated" API that was on every SVGA card in the middle of the 90s ? They make a huge difference and was well supported on Windows and X11
  - pcwalton 3y ago
    That stuff has been obsolete for quite a while as the general 3D capabilities are more than enough to saturate all the GPU's memory bandwidth.
    - zozbot234 3y ago
      If it's a VESA standard and still supported it might be useful as a fallback for hardware that doesn't have its own driver. Edit: But actually, I couldn't find references to anything similar besides VBE/AF which even when current got almost no support directly in hardware, so folks had to resort to hardware-specific DOS TSR's. I'm not sure if there's anything newer than that.
  - soundarana 3y ago
    The basic display interface used by UEFI and low level boot loaders these days it's called GOP - Graphics Output Protocol. It replaced VESA.
  - rjsw 3y ago
    GPU manufacturers stopped putting 2D functionality in their chips.
sprash 3y ago
Xrender is hardware accelerated and cairo uses Xrender as a backend. Why is Xrender not "viable"?
- pcwalton 3y ago
  Honestly I think XRENDER could be a viable API--the core idea is similar to WebRender, which Firefox uses to great effect--but the existing implementations of it are not well-optimized implementations and issue tons of draw calls using obsolete OpenGL APIs. They are slower than just drawing on CPU. You would essentially need a complete rewrite. The bigger issue is that there's little reason to farm vector graphics rendering out to the window server in the first place. The main reason would be to avoid a window blit on HiDPI displays. But the tradeoff is that the XRENDER API is all you get, and usually apps have more sophisticated needs than what it can provide. For instance, browsers can't really use XRENDER nowadays because there's no way to describe CSS 3D transforms in it. And if you use it you're at the mercy of the window server to implement it reasonably, which is not a safe assumption. (A lot of the reason Chrome on Linux was faster than Firefox in the early days is that Firefox used XRENDER, while Chrome rendered on CPU. I remember at least one engineer at Mozilla who was bitter about that, after putting in all the work to make Firefox use it only to have it be a net loss.) In any case, you can avoid the window blit by simply using scanout compositing, as detailed in my other reply, so there is really is no compelling reason to reinvent XRENDER.
- Jasper_ 3y ago
  Xrender is hardware-accelerated on an increasingly small number of devices, and even SNA, the flagship hardware-accelerated implementation in the Intel driver fell back to software rasterization extremely frequently [0]. In practice it wasn't worth it, and it was extremely buggy, hence why it fell into disrepair. The semantics of Xrender simply don't match with what modern GPUs give you, even ones with 2D pipelines. [0] https://gitlab.freedesktop.org/search?search=sna_pixmap_move_to_cpu&nav_source=navbar&project_id=612&group_id=2215&search_code=true&repository_ref=master https://gitlab.freedesktop.org/search?search=sna_pixmap_move...
Dwedit 3y ago
Well there was once a hardware accelerated API for 2D drawing on Windows (DirectDraw), but it died in Windows Vista when desktop composition was added in. It was still supported for application use, but it was just emulating it. But if there was an API for 2D acceleration that was actually supported (and could be used simultaneously with desktop composition), then it could be added in to something like SDL then suddenly applications would support it.
- anthk 3y ago
  It's slow as hell, today you need to use WIneD3D's Ddraw.dll among the WineD3D loader in the same folder of your 2D game.

pcwalton 3y ago

For the most part that's true, but simple 2D compositing is a bit of a different beast, because it can sometimes be done at scanout time, saving a blit. Last I checked, (non-Android) Linux rarely makes use of this except for the mouse cursor. But in general you can save a good bit of energy and memory bandwidth on HiDPI displays if you try to use 2D hardware layers where you can. You can virtually never use them for the UI itself, because they're far too limited, but the windowing system can often use them to composite windows together. It'd be nice if Wayland compositors made more use of this, e.g. to avoid having to blit the foreground window every frame.

kimixa 3y ago
I used to work on mobile graphics and the android HWC stack. The scanout-time hardware was often less useful that you might think - only in dynamic scenes where the GPU is otherwise idle (like playing video possibly with a static UI overlay was the premier use case). For static scenes it's more efficient to render out to a buffer (using the GPU as the scanout overlay pipes often had limited feedback capability) and just output that using overlays disabled. It didn't take many frames for that to be worth it. For apps that were animating or otherwise updating it's window, most UI toolkits used the GPU for widget rendering. And often the scanout pipes didn't hook into the (relatively large) system caches like the GPU did, so there were times it was again faster to composite the screen on the GPU to a single scanout buffer than flush already cached data, the get the scanout hardware to read it back from the memory bus. And there weren't as cheap as people thought - one stat I remember was that the total area of the GPU on the omap4 platform was smaller than the display pipes. Though that is now a pretty old chip, and always had a bit of focus on "multimedia".
- deleted 3y ago
  [deleted]
- kllrnohj 3y ago
  I think your information is quite outdated. The HWC overlay planes are heavily used, you can see this trivially just doing a 'dumpsys SurfaceFlinger' or grabbing a systrace/perfetto trace. When it falls back to GPU composition it's very obvious as there's a significant hit to latency and more GPU contention. The overlay capabilities of the modern Snapdragons are also quite absurd. They support like upwards of a dozen overlays now and even have FP16 extended sRGB support. Some HWCs (like the one in the steam deck) even have per plane 3D LUTs for HDR tone mapping (ex https://github.com/ValveSoftware/gamescope/blob/master/src/docs/Steam%20Deck%20Display%20Pipeline.png https://github.com/ValveSoftware/gamescope/blob/master/src/d... ) The composition is bandwidth heavy of course, but for static scenes there's a cache after the HWC in the form of panel self refresh.
Jasper_ 3y ago
CRTC planes and scanout-time compositing makes sense, and Wayland compositors do use them, even for non-cursor surfaces. It's simply not something an application can use general-purpose and guarantee (though see the recent GtkSurfaceOffload stuff for the latest attempt at it). Personally, I don't see it as a "2D drawing API", it doesn't accelerate anything special about 2D, only blits and transforms, which a 3D API will eat for breakfast.
Zardoz84 3y ago
what happened with these VESA "2d accelerated" API that was on every SVGA card in the middle of the 90s ? They make a huge difference and was well supported on Windows and X11
- pcwalton 3y ago
  That stuff has been obsolete for quite a while as the general 3D capabilities are more than enough to saturate all the GPU's memory bandwidth.
  - zozbot234 3y ago
    If it's a VESA standard and still supported it might be useful as a fallback for hardware that doesn't have its own driver. Edit: But actually, I couldn't find references to anything similar besides VBE/AF which even when current got almost no support directly in hardware, so folks had to resort to hardware-specific DOS TSR's. I'm not sure if there's anything newer than that.
- soundarana 3y ago
  The basic display interface used by UEFI and low level boot loaders these days it's called GOP - Graphics Output Protocol. It replaced VESA.
- rjsw 3y ago
  GPU manufacturers stopped putting 2D functionality in their chips.

sprash 3y ago

Xrender is hardware accelerated and cairo uses Xrender as a backend. Why is Xrender not "viable"?

pcwalton 3y ago
Honestly I think XRENDER could be a viable API--the core idea is similar to WebRender, which Firefox uses to great effect--but the existing implementations of it are not well-optimized implementations and issue tons of draw calls using obsolete OpenGL APIs. They are slower than just drawing on CPU. You would essentially need a complete rewrite. The bigger issue is that there's little reason to farm vector graphics rendering out to the window server in the first place. The main reason would be to avoid a window blit on HiDPI displays. But the tradeoff is that the XRENDER API is all you get, and usually apps have more sophisticated needs than what it can provide. For instance, browsers can't really use XRENDER nowadays because there's no way to describe CSS 3D transforms in it. And if you use it you're at the mercy of the window server to implement it reasonably, which is not a safe assumption. (A lot of the reason Chrome on Linux was faster than Firefox in the early days is that Firefox used XRENDER, while Chrome rendered on CPU. I remember at least one engineer at Mozilla who was bitter about that, after putting in all the work to make Firefox use it only to have it be a net loss.) In any case, you can avoid the window blit by simply using scanout compositing, as detailed in my other reply, so there is really is no compelling reason to reinvent XRENDER.
Jasper_ 3y ago
Xrender is hardware-accelerated on an increasingly small number of devices, and even SNA, the flagship hardware-accelerated implementation in the Intel driver fell back to software rasterization extremely frequently [0]. In practice it wasn't worth it, and it was extremely buggy, hence why it fell into disrepair. The semantics of Xrender simply don't match with what modern GPUs give you, even ones with 2D pipelines. [0] https://gitlab.freedesktop.org/search?search=sna_pixmap_move_to_cpu&nav_source=navbar&project_id=612&group_id=2215&search_code=true&repository_ref=master https://gitlab.freedesktop.org/search?search=sna_pixmap_move...

Dwedit 3y ago

Well there was once a hardware accelerated API for 2D drawing on Windows (DirectDraw), but it died in Windows Vista when desktop composition was added in. It was still supported for application use, but it was just emulating it. But if there was an API for 2D acceleration that was actually supported (and could be used simultaneously with desktop composition), then it could be added in to something like SDL then suddenly applications would support it.

anthk 3y ago
It's slow as hell, today you need to use WIneD3D's Ddraw.dll among the WineD3D loader in the same folder of your 2D game.