4 ms·

- Shouldn't 2D rendering be a solved problem given that it's basically a subset of 3D rendering? - Don't libraries like Skia, Qt, Cairo use GPU rendering? I've

- Shouldn't 2D rendering be a solved problem given that it's basically a subset of 3D rendering? - Don't libraries like Skia, Qt, Cairo use GPU rendering? I've always assumed so. I mean, this is 2020, GPUs have been around for decades.

pcwalton 6y ago
> - Shouldn't 2D rendering be a solved problem given that it's basically a subset of 3D rendering? The problem is that primitives artists use are different. 3D rendering tends to all consist of polygon meshes, which are relatively easy to render. 2D rendering (basically) consists of Bezier paths, which are harder. The equivalent in 3D, which is adaptive subdivision, is not really a solved problem in real-time either. Additionally, 2D rendering quality tends to be more important than 3D rendering quality. Whereas you can get away with 4xMSAA or hacks like FXAA in 3D, true 16xAA (without hacks) is the absolute minimum for 2D rendering quality nowadays, and even it isn't considered great for some tasks like font rendering (Pathfinder and piet-gpu both use analytic AA which is effectively 256xAA). > - Don't libraries like Skia, Qt, Cairo use GPU rendering? I've always assumed so. I mean, this is 2020, GPUs have been around for decades. There's a difference between renderers with GPU support and renderers that are oriented around using the GPU efficiently. In many cases this results in an order-of-magnitude speedup. On the GPU, state changes are expensive, and many such renderers that have GPU support don't really go out of their way to avoid them. There are also occlusion culling optimizations that most renderers don't do, but piet-gpu and Pathfinder do.
stanfordkid 6y ago
Things like multi-resolution font-rendering are actually more complicated than one might imagine. The easy way is to just tesselate the font into polygons -- but this tesselation often depends on the zoom level. The same thing can be said about implicit curves etc. Most libraries do use GPU's for basic draw operations (e.g rendering a gradient), but to build something like Photoshop -- you need much more complexity.
raphlinus 6y ago
Others have spoken to this, but as a general introduction I highly recommend Jasper's post "Why are 2D vector graphics so much harder than 3D?" In short, no, it's not a solved problem. Also, I see a lot of variations of this question, but I should state this more clearly. There's been accelerated graphics in one form or another for a long time, but what I'm doing is a completely different type of thing. In my world, on the CPU you just encode the scene into a binary representation that's optimized for GPU but in many ways is like flatbuffers, and then the GPU runs a highly parallel program to render the whole thing. In previous approaches, the CPU is deeply involved in taking the scene apart and putting it back together in a form that's well suited to relatively dumb pixel pipes. Now that GPUs are really fast, that approach runs into limitations. It also depends what you're trying to do. I'm focusing here on dynamic paths (and thus font rendering), while most of the libraries optimized for UI put text into texture atlases and then use the GPU to composite quads to the final surface, something they can do well. https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harder-than-3d-graphics/ https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harde...
- raks435 6y ago
  Can you expound the principle of tiling mentioned in your algorithm a bit more ? The conventional mechanism is to use de-Casteljau to divide a bezier curve into triangles and then rasterize these triangles using GPU. If the curve is required to be scaled, the triangularization/tesselation is done again How is the algorithm presented in the link different ? Somehow the concept of tiling seems to imply that rasterization of the curve is done in the CPU itself. What am I missing ?
  - raphlinus 6y ago
    I recommend reading the blog post series, I'm not sure I can usefully summarize the concepts in a comment reply. But very briefly, there's a flattening step (evaluated on GPU, based on de Casteljau) that converts the Bezier into a polyline (not triangles), then a tiling step that records for each tile a "command list" that contains the complete description of how to render the pixels in a tile, finally "fine rasterization" so that each workgroup reads that command list and renders 256 pixels in parallel from it. From your question, it sounds like your mental model is pretty different from how this pipeline works.
jcelerier 6y ago
> Don't libraries like Skia, Qt, Cairo use GPU rendering? I've always assumed so. I mean, this is 2020, GPUs have been around for decades. As a Qt user / developer - you can use the GPU for your app, e.g. with Qt Quick or QGraphicsView, but there are sometimes good reasons to stick to CPU & software rendering, e.g. it is somewhat common to want to have intertwined "native" OS widgets (which are all CPU-rendered raster things) and custom GPU-drawn scene - this is a case where things fall a bit apart. Another thing is that pretty, freetype-like font rendering is super expensive when you have a lot of text to show, and can't really be done (at least I have definitely not seen infinality-level beauty from the state of the art) on the GPU yet... next to that filling some rects (read: 95% of UI) with SSE/AVX/AVX2 as Qt does is stupidly fast.
- chrismorgan 6y ago
  > pretty, freetype-like font rendering is super expensive when you have a lot of text to show, and can't really be done (at least I have definitely not seen infinality-level beauty from the state of the art) on the GPU yet This is exactly what Pathfinder does. pcwalton is on this thread and is the main author of that. https://github.com/servo/pathfinder#features https://github.com/servo/pathfinder#features says: > Advanced font rendering. Pathfinder can render fonts with slight hinting and can perform subpixel antialiasing on LCD screens. It can do stem darkening/font dilation like macOS and FreeType in order to make text easier to read at small sizes. The library also has support for gamma correction.
  - jcelerier 6y ago
    > This is exactly what Pathfinder does. from the screenshots I saw so far, pretty much not. > It can do stem darkening/font dilation like macOS and FreeType in order to make text easier to read at small sizes. there are tons of different ways to do that. Even freetype has a few different algorithms to do it, some not even merged if I'm not mistaken, which give wildly different results
    - chrismorgan 6y ago
      To give you an idea that pcwalton knows what he’s been doing and has indeed been seeking to match platform rendering exactly, here are a couple of tweets about the macOS font dilation: https://twitter.com/pcwalton/status/918593367914803201 https://twitter.com/pcwalton/status/918593367914803201, https://twitter.com/pcwalton/status/918991457532354560 https://twitter.com/pcwalton/status/918991457532354560. I rather like the demonstration of rendering including subpixel rendering at https://twitter.com/pcwalton/status/971475785616797698 https://twitter.com/pcwalton/status/971475785616797698, as well.
pritovido 6y ago
No, it is not a subset, unless you are talking about drawing textures. Those libraries use GPU rendering using textures, like distance fields(Qt) or simply calculate fonts in 2D and draw it on textures using the GPU, like Apple or cairo usually do. Distance fields are blurry when you have small fonts on display. Things like calculating the exact area order the 2D curve is something you could easily do in CPU, but it is extremely difficult to do on the GPU. You need decoupled data in order to parallelize it.

pcwalton 6y ago

> - Shouldn't 2D rendering be a solved problem given that it's basically a subset of 3D rendering? The problem is that primitives artists use are different. 3D rendering tends to all consist of polygon meshes, which are relatively easy to render. 2D rendering (basically) consists of Bezier paths, which are harder. The equivalent in 3D, which is adaptive subdivision, is not really a solved problem in real-time either. Additionally, 2D rendering quality tends to be more important than 3D rendering quality. Whereas you can get away with 4xMSAA or hacks like FXAA in 3D, true 16xAA (without hacks) is the absolute minimum for 2D rendering quality nowadays, and even it isn't considered great for some tasks like font rendering (Pathfinder and piet-gpu both use analytic AA which is effectively 256xAA). > - Don't libraries like Skia, Qt, Cairo use GPU rendering? I've always assumed so. I mean, this is 2020, GPUs have been around for decades. There's a difference between renderers with GPU support and renderers that are oriented around using the GPU efficiently. In many cases this results in an order-of-magnitude speedup. On the GPU, state changes are expensive, and many such renderers that have GPU support don't really go out of their way to avoid them. There are also occlusion culling optimizations that most renderers don't do, but piet-gpu and Pathfinder do.

stanfordkid 6y ago

Things like multi-resolution font-rendering are actually more complicated than one might imagine. The easy way is to just tesselate the font into polygons -- but this tesselation often depends on the zoom level. The same thing can be said about implicit curves etc. Most libraries do use GPU's for basic draw operations (e.g rendering a gradient), but to build something like Photoshop -- you need much more complexity.

raphlinus 6y ago

Others have spoken to this, but as a general introduction I highly recommend Jasper's post "Why are 2D vector graphics so much harder than 3D?" In short, no, it's not a solved problem. Also, I see a lot of variations of this question, but I should state this more clearly. There's been accelerated graphics in one form or another for a long time, but what I'm doing is a completely different type of thing. In my world, on the CPU you just encode the scene into a binary representation that's optimized for GPU but in many ways is like flatbuffers, and then the GPU runs a highly parallel program to render the whole thing. In previous approaches, the CPU is deeply involved in taking the scene apart and putting it back together in a form that's well suited to relatively dumb pixel pipes. Now that GPUs are really fast, that approach runs into limitations. It also depends what you're trying to do. I'm focusing here on dynamic paths (and thus font rendering), while most of the libraries optimized for UI put text into texture atlases and then use the GPU to composite quads to the final surface, something they can do well. https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harder-than-3d-graphics/ https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harde...

raks435 6y ago
Can you expound the principle of tiling mentioned in your algorithm a bit more ? The conventional mechanism is to use de-Casteljau to divide a bezier curve into triangles and then rasterize these triangles using GPU. If the curve is required to be scaled, the triangularization/tesselation is done again How is the algorithm presented in the link different ? Somehow the concept of tiling seems to imply that rasterization of the curve is done in the CPU itself. What am I missing ?
- raphlinus 6y ago
  I recommend reading the blog post series, I'm not sure I can usefully summarize the concepts in a comment reply. But very briefly, there's a flattening step (evaluated on GPU, based on de Casteljau) that converts the Bezier into a polyline (not triangles), then a tiling step that records for each tile a "command list" that contains the complete description of how to render the pixels in a tile, finally "fine rasterization" so that each workgroup reads that command list and renders 256 pixels in parallel from it. From your question, it sounds like your mental model is pretty different from how this pipeline works.

jcelerier 6y ago

> Don't libraries like Skia, Qt, Cairo use GPU rendering? I've always assumed so. I mean, this is 2020, GPUs have been around for decades. As a Qt user / developer - you can use the GPU for your app, e.g. with Qt Quick or QGraphicsView, but there are sometimes good reasons to stick to CPU & software rendering, e.g. it is somewhat common to want to have intertwined "native" OS widgets (which are all CPU-rendered raster things) and custom GPU-drawn scene - this is a case where things fall a bit apart. Another thing is that pretty, freetype-like font rendering is super expensive when you have a lot of text to show, and can't really be done (at least I have definitely not seen infinality-level beauty from the state of the art) on the GPU yet... next to that filling some rects (read: 95% of UI) with SSE/AVX/AVX2 as Qt does is stupidly fast.

chrismorgan 6y ago
> pretty, freetype-like font rendering is super expensive when you have a lot of text to show, and can't really be done (at least I have definitely not seen infinality-level beauty from the state of the art) on the GPU yet This is exactly what Pathfinder does. pcwalton is on this thread and is the main author of that. https://github.com/servo/pathfinder#features https://github.com/servo/pathfinder#features says: > Advanced font rendering. Pathfinder can render fonts with slight hinting and can perform subpixel antialiasing on LCD screens. It can do stem darkening/font dilation like macOS and FreeType in order to make text easier to read at small sizes. The library also has support for gamma correction.
- jcelerier 6y ago
  > This is exactly what Pathfinder does. from the screenshots I saw so far, pretty much not. > It can do stem darkening/font dilation like macOS and FreeType in order to make text easier to read at small sizes. there are tons of different ways to do that. Even freetype has a few different algorithms to do it, some not even merged if I'm not mistaken, which give wildly different results
  - chrismorgan 6y ago
    To give you an idea that pcwalton knows what he’s been doing and has indeed been seeking to match platform rendering exactly, here are a couple of tweets about the macOS font dilation: https://twitter.com/pcwalton/status/918593367914803201 https://twitter.com/pcwalton/status/918593367914803201, https://twitter.com/pcwalton/status/918991457532354560 https://twitter.com/pcwalton/status/918991457532354560. I rather like the demonstration of rendering including subpixel rendering at https://twitter.com/pcwalton/status/971475785616797698 https://twitter.com/pcwalton/status/971475785616797698, as well.

pritovido 6y ago

No, it is not a subset, unless you are talking about drawing textures. Those libraries use GPU rendering using textures, like distance fields(Qt) or simply calculate fonts in 2D and draw it on textures using the GPU, like Apple or cairo usually do. Distance fields are blurry when you have small fonts on display. Things like calculating the exact area order the 2D curve is something you could easily do in CPU, but it is extremely difficult to do on the GPU. You need decoupled data in order to parallelize it.