5 ms·

It is. There's a terminology problem in play here. Throughout this article 2D does not mean 2D, it means "arbitrary complexity paths, eg bezier curves". This is

It is. There's a terminology problem in play here. Throughout this article 2D does not mean 2D, it means "arbitrary complexity paths, eg bezier curves". This is a small subset of 2D as makes up a UI. It'd be like saying 3D exclusively means infinitely detailed tesellated shapes with path traced rendering. That's definitely an area of 3D that exists, but of course is not at all the entirety of 3D in practice in eg games or movies. Rather it's more like the holy grail. Same thing here with eg. SVGs. GPU accelerating SVGs is stupidly complicated because it's an inherently serial algorithm, and GPUs are poop at that. But how much of your 2D UI is made up of that? Text is in that same category but how much else? Typically very little. Maybe a few icons, but that tends to be about it. Instead you have higher level shapes, like round rectangles. And those you can do with a GPU quite easily. Similarly images are usually just a textured quad. Again trivial for a GPU. You could describe them as paths if you had a fully generic, fully accelerated path rendering system. But nobody has that, so nobody actually describes them like that. So very nearly all 2D/UI systems are GPU accelerated. It'd be perhaps more accurate to call them hybrid renderers. Things like fonts are just CPU rendered because CPUs are better at it, but the GPU is doing all the fills, gradients, "simple" shapes, texturing, etc...

Jasper_ 5y ago
I don't know if I'd go that far -- icons and text are vector paths. Strokes and drop shadows (aka blurs) are all things that GPUs aren't great particularly great at. Simple shapes like rounded rectangles, GPUs can be OK at, but you'd have overdraw problems if done naively. I've worked on 2D rendering engines, so I've seen the content thrown at it in the wild. Very rarely do you have a simple case. GitHub's buttons are maybe the simplest example I can think of, and they have strokes (GPUs: ugh) on a filled rounded border (GPUs: ugh), with text inside (GPUs: ugh), sometimes with a text shadow (GPUs: ugh). It can be done, but you basically have to get away from triangles and move into research methods which are exceptionally more tricky, aka the stuff in Pathfinder and piet-gpu.
- teknopaul 5y ago
  Showing my ignorance here: Why we don't have dedicated graphics hardware for 2D? Feels like moving 2D jobs to a fast 3D engine is wrong approach at the top level.
  - zozbot234 5y ago
    "Dedicated graphics hardware for 2D" is just a blitter and sprite engine, and modern GPU's do that just fine. The mouse pointer in many recent systems is a 2D hardware sprite.
- kllrnohj 5y ago
  > Strokes and drop shadows (aka blurs) are all things that GPUs aren't great particularly great at. They can handle those just fine. Blurs are just inherently very expensive, but GPUs are no worse at them than CPUs. In fact GPUs are way faster at blurs than CPUs. Same with filled shapes. It's not really a challenge. You have a fragment shader that knows how to essentially 'clip' to a round rect, which isn't hard, and then filling it any which way with anything is trivial.
raphlinus 5y ago
I respectfully disagree that CPUs are better than GPUs at font rendering :) There are a few related things that can be said. Doing fast, high quality font rendering on a GPU is hard; it's much easier on a CPU. Further, the traditional rasterization pipeline of a GPU is not good at rendering fonts. Fortunately, modern GPUs also have compute shaders, which are programmed somewhat like regular computers but just with an astonishingly high number of threads. This is the topic of my research, and I intend to publish quantitative measurements backing up these assertions before long. Early results look promising.
- danielvaughn 5y ago
  You probably know about Slug, but just in case: https://sluglibrary.com https://sluglibrary.com I don't know much about the font space, but enough to know it's a really hard problem, and the slug team seems to do a really good job.
  - raphlinus 5y ago
    Absolutely, and I don't want to claim I'm the first or only one doing font rendering on GPU. There's Slug as you pointed out, Pathfinder and Spinel as Jasper cited, and also interesting experimental work including GLyphy by Behdad and algorithms by Evan Wallace and Will Dobbie, plus a whole series of academic papers including "Massively Parallel Vector Graphics," "Random Access Vector Graphics," and others. However, I would say that a common thread is that doing this well is hard. There's no straightforward cookbook scheme that people can just implement, and there are always tradeoffs. Slug is used in a number of games (and congrats to Eric for winning those licenses), but not as far as I know in any UI toolkits, and there are reasons for that.
    - moonchild 5y ago
      > Slug is used in a number of games (and congrats to Eric for winning those licenses), but not as far as I know in any UI toolkits, and there are reasons for that Presumably because its antialiasing is crap? But there's nothing inherent to fragment-oriented approaches that prevents you from doing good aa, and they slot nicely into the existing rasterization pipeline (which is why slug has fewer feature level requirements than pathfinder). They also permit arbitrary domain transformations (some caveats here as you have to calculate a bounding box still), and given appropriate space partitioning should not be significantly slower than scanline algorithms. Also: UI toolkits are not known for being on the leading edge of graphics research. I think fastuidraw demonstrates this rather well. Insofar as there is exciting work happening in industry, it is mainly happening in web browsers; and I would expect mozilla and google to devote their efforts pathfinder and skia, respectively.

Jasper_ 5y ago

I don't know if I'd go that far -- icons and text are vector paths. Strokes and drop shadows (aka blurs) are all things that GPUs aren't great particularly great at. Simple shapes like rounded rectangles, GPUs can be OK at, but you'd have overdraw problems if done naively. I've worked on 2D rendering engines, so I've seen the content thrown at it in the wild. Very rarely do you have a simple case. GitHub's buttons are maybe the simplest example I can think of, and they have strokes (GPUs: ugh) on a filled rounded border (GPUs: ugh), with text inside (GPUs: ugh), sometimes with a text shadow (GPUs: ugh). It can be done, but you basically have to get away from triangles and move into research methods which are exceptionally more tricky, aka the stuff in Pathfinder and piet-gpu.

teknopaul 5y ago
Showing my ignorance here: Why we don't have dedicated graphics hardware for 2D? Feels like moving 2D jobs to a fast 3D engine is wrong approach at the top level.
- zozbot234 5y ago
  "Dedicated graphics hardware for 2D" is just a blitter and sprite engine, and modern GPU's do that just fine. The mouse pointer in many recent systems is a 2D hardware sprite.
kllrnohj 5y ago
> Strokes and drop shadows (aka blurs) are all things that GPUs aren't great particularly great at. They can handle those just fine. Blurs are just inherently very expensive, but GPUs are no worse at them than CPUs. In fact GPUs are way faster at blurs than CPUs. Same with filled shapes. It's not really a challenge. You have a fragment shader that knows how to essentially 'clip' to a round rect, which isn't hard, and then filling it any which way with anything is trivial.

raphlinus 5y ago

I respectfully disagree that CPUs are better than GPUs at font rendering :) There are a few related things that can be said. Doing fast, high quality font rendering on a GPU is hard; it's much easier on a CPU. Further, the traditional rasterization pipeline of a GPU is not good at rendering fonts. Fortunately, modern GPUs also have compute shaders, which are programmed somewhat like regular computers but just with an astonishingly high number of threads. This is the topic of my research, and I intend to publish quantitative measurements backing up these assertions before long. Early results look promising.

danielvaughn 5y ago
You probably know about Slug, but just in case: https://sluglibrary.com https://sluglibrary.com I don't know much about the font space, but enough to know it's a really hard problem, and the slug team seems to do a really good job.
- raphlinus 5y ago
  Absolutely, and I don't want to claim I'm the first or only one doing font rendering on GPU. There's Slug as you pointed out, Pathfinder and Spinel as Jasper cited, and also interesting experimental work including GLyphy by Behdad and algorithms by Evan Wallace and Will Dobbie, plus a whole series of academic papers including "Massively Parallel Vector Graphics," "Random Access Vector Graphics," and others. However, I would say that a common thread is that doing this well is hard. There's no straightforward cookbook scheme that people can just implement, and there are always tradeoffs. Slug is used in a number of games (and congrats to Eric for winning those licenses), but not as far as I know in any UI toolkits, and there are reasons for that.
  - moonchild 5y ago
    > Slug is used in a number of games (and congrats to Eric for winning those licenses), but not as far as I know in any UI toolkits, and there are reasons for that Presumably because its antialiasing is crap? But there's nothing inherent to fragment-oriented approaches that prevents you from doing good aa, and they slot nicely into the existing rasterization pipeline (which is why slug has fewer feature level requirements than pathfinder). They also permit arbitrary domain transformations (some caveats here as you have to calculate a bounding box still), and given appropriate space partitioning should not be significantly slower than scanline algorithms. Also: UI toolkits are not known for being on the leading edge of graphics research. I think fastuidraw demonstrates this rather well. Insofar as there is exciting work happening in industry, it is mainly happening in web browsers; and I would expect mozilla and google to devote their efforts pathfinder and skia, respectively.