6 ms·

GPU accelerated vector graphics rendering is an open problem.

by duped 4y ago

naikrovek 4y ago
is it? MacOS has rendered the entire UI via PDF since OSX was first released.
- danielvaughn 4y ago
  afaik there just hasn’t been a lot of work done towards leveraging the GPU for vector graphics. The one example I can think of is the Slug library.
  - djbusby 4y ago
    Like? SVG UIs everywhere? Could you make a UI that's like SVG and onclick handlers do the "magic". We could have a VB6 like SVG UI designer and program "actions" in $LANG?
    - ygra 4y ago
      WPF would probably be that. In any case, modern UI toolkits often are vector-based anyway because of UI scaling. Heck, even ancient ones are, if you look at the Marlett font. That contained scalable UI elements for Windows 95.
  - ehvatum 4y ago
    > afaik there just hasn’t been a lot of work done towards leveraging the GPU for vector graphic Sure there has been, in the form of the nvidia path rendering extension. https://developer.nvidia.com/nv-path-rendering https://developer.nvidia.com/nv-path-rendering It works great, on nvidia cards. Adobe Illustrator on Windows makes good use of it. Calling the extension from C/C++ and Python is no big deal.
- astrange 4y ago
  It's not really PDF, it just supports similar operations to PDF. There's more SVG UI elements now.
  - naikrovek 4y ago
    but it’s vector graphics, right?
    - astrange 4y ago
      Only very recently. Like, all that old Aqua stuff wasn't vectors, it was done in Photoshop. Bitmaps are much faster to draw and can be made pixel-precise up front. With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
      - naikrovek 4y ago
        > Bitmaps are much faster to draw and can be made pixel-precise up front. yes, but that alone doesn't require that bitmap graphics be used. > With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px. only if your renderer or your artwork isn't done correctly; inkscape used to be notorious for floating point rounding errors showing up with very few manipulations of a shape. it's also been very easy to very quickly rasterize vector graphics for some time. maybe not fast enough in the past I guess, but computers are certainly fast enough now. I look forward to a time when UIs are just all vector all the time, and it's the monitor itself which rasterizes, thanks to a program that the attached device uploads as it boots up or as the monitor is connected. it's going to get more and more difficult to continue increasing display resolutions and refresh rates over long inexpensive cables if we continue to rasterize on a computer and push raw pixels to the display device.
raphlinus 4y ago
For now. Unfortunately, even if you have very fast drawing, there are plenty of other ways to make software slow. But high performance GUI software is the main thing I'm working on at the moment and I expect to have interesting results in the coming months. If nothing else, it should be a data point of how fast GUI software can be.
- danielvaughn 4y ago
  I’m super excited to hear about this. I’ve been working on a programming language for UI designers, and I ran into some massive issues when I tried to build my own renderer. Gave up and now I’m just transpiling to other languages instead.
- jahewson 4y ago
  Eagerly awaiting this!
jiggawatts 4y ago
It isn’t an open problem for text editors or terminals, which is the point Casey made in his videos.
- naikrovek 4y ago
  text editors and terminals don’t render vector graphics though, do they? they rely on libraries to rasterize glyphs of a font into a bitmap once and use the bitmaps over and over.
  - _gabe_ 4y ago
    > they rely on libraries to rasterize glyphs of a font into a bitmap once Unfortunately it's not this simple. Unicode and multilingual support means you can't possibly generate a single bitmap to contain all possible characters. Modern UIs (including text editors and terminals) generally support various zoom levels, which requires re-rasterizing glyphs at different zoom levels. This means you typically have to rasterize vector graphics (your font) on the fly as needed and then update your bitmap. Edit: Also, emojis and certain scripts require composting multiple glyphs together. This means you have to figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately. Then throw in mathematical notation and all bets are off. My point is, rendering text is not a simple done deal, but it's also not an impossible feat and it can be done efficiently.
    - naikrovek 4y ago
      that is a very pedantic argument. you completely overlooked the point i was trying to make. vector graphics are not rendered directly to the screen. fonts are rasterized first, then the rasterization is what the GPU receives.
      - _gabe_ 4y ago
        > vector graphics are not rendered directly to the screen. What difference does it make if you're caching the rasterization or not? You're still rendering vector graphics. And you're not just rasterizing once. Modern font renderers can't get away with rasterizing glyphs once and reusing them anymore than arbitrary vector graphics renderers can. I sure hope my vector graphics editors cache the rasterizations instead of re-rasterizing needlessly.
rayiner 4y ago
But you don’t have to do it every frame. Most apps could just render to bitmaps once and cache the results.
userbinator 4y ago
You shouldn't even need a GPU for basic UI like this, especially with "modern" ultra-minimalist flat UIs.
- vladvasiliu 4y ago
  Yeah, but they're not actually completely flat since they're translucent. Also, on Windows, the "acrylic" thing is a circus show. Every app behaves differently. It also manages to sometimes lag on an RX5600. I know it's an "old" gpu, but still...
jcelerier 4y ago
Maybe basic productivity applications shouldn't try to tackle open research problems and instead just do the thing which has worked fine for the last 30 years which is CPU rendering? Like, do you think notepad is going to burn your battery because it redraws 20x20 pixels on every mouse event ?
moonchild 4y ago
It is being worked on (ahem...) But in most contexts, it is not necessary to render vector graphics 'live'; caching is acceptable, so this should not be a bottleneck for most desktop software.

GPU accelerated vector graphics rendering is an open problem.

by duped 4y ago

GPU accelerated vector graphics rendering is an open problem.

naikrovek 4y ago

is it? MacOS has rendered the entire UI via PDF since OSX was first released.

danielvaughn 4y ago
afaik there just hasn’t been a lot of work done towards leveraging the GPU for vector graphics. The one example I can think of is the Slug library.
- djbusby 4y ago
  Like? SVG UIs everywhere? Could you make a UI that's like SVG and onclick handlers do the "magic". We could have a VB6 like SVG UI designer and program "actions" in $LANG?
  - ygra 4y ago
    WPF would probably be that. In any case, modern UI toolkits often are vector-based anyway because of UI scaling. Heck, even ancient ones are, if you look at the Marlett font. That contained scalable UI elements for Windows 95.
- ehvatum 4y ago
  > afaik there just hasn’t been a lot of work done towards leveraging the GPU for vector graphic Sure there has been, in the form of the nvidia path rendering extension. https://developer.nvidia.com/nv-path-rendering https://developer.nvidia.com/nv-path-rendering It works great, on nvidia cards. Adobe Illustrator on Windows makes good use of it. Calling the extension from C/C++ and Python is no big deal.
astrange 4y ago
It's not really PDF, it just supports similar operations to PDF. There's more SVG UI elements now.
- naikrovek 4y ago
  but it’s vector graphics, right?
  - astrange 4y ago
    Only very recently. Like, all that old Aqua stuff wasn't vectors, it was done in Photoshop. Bitmaps are much faster to draw and can be made pixel-precise up front. With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
    - naikrovek 4y ago
      > Bitmaps are much faster to draw and can be made pixel-precise up front. yes, but that alone doesn't require that bitmap graphics be used. > With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px. only if your renderer or your artwork isn't done correctly; inkscape used to be notorious for floating point rounding errors showing up with very few manipulations of a shape. it's also been very easy to very quickly rasterize vector graphics for some time. maybe not fast enough in the past I guess, but computers are certainly fast enough now. I look forward to a time when UIs are just all vector all the time, and it's the monitor itself which rasterizes, thanks to a program that the attached device uploads as it boots up or as the monitor is connected. it's going to get more and more difficult to continue increasing display resolutions and refresh rates over long inexpensive cables if we continue to rasterize on a computer and push raw pixels to the display device.

raphlinus 4y ago

For now. Unfortunately, even if you have very fast drawing, there are plenty of other ways to make software slow. But high performance GUI software is the main thing I'm working on at the moment and I expect to have interesting results in the coming months. If nothing else, it should be a data point of how fast GUI software can be.

danielvaughn 4y ago
I’m super excited to hear about this. I’ve been working on a programming language for UI designers, and I ran into some massive issues when I tried to build my own renderer. Gave up and now I’m just transpiling to other languages instead.
jahewson 4y ago
Eagerly awaiting this!

jiggawatts 4y ago

It isn’t an open problem for text editors or terminals, which is the point Casey made in his videos.

naikrovek 4y ago
text editors and terminals don’t render vector graphics though, do they? they rely on libraries to rasterize glyphs of a font into a bitmap once and use the bitmaps over and over.
- _gabe_ 4y ago
  > they rely on libraries to rasterize glyphs of a font into a bitmap once Unfortunately it's not this simple. Unicode and multilingual support means you can't possibly generate a single bitmap to contain all possible characters. Modern UIs (including text editors and terminals) generally support various zoom levels, which requires re-rasterizing glyphs at different zoom levels. This means you typically have to rasterize vector graphics (your font) on the fly as needed and then update your bitmap. Edit: Also, emojis and certain scripts require composting multiple glyphs together. This means you have to figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately. Then throw in mathematical notation and all bets are off. My point is, rendering text is not a simple done deal, but it's also not an impossible feat and it can be done efficiently.
  - naikrovek 4y ago
    that is a very pedantic argument. you completely overlooked the point i was trying to make. vector graphics are not rendered directly to the screen. fonts are rasterized first, then the rasterization is what the GPU receives.
    - _gabe_ 4y ago
      > vector graphics are not rendered directly to the screen. What difference does it make if you're caching the rasterization or not? You're still rendering vector graphics. And you're not just rasterizing once. Modern font renderers can't get away with rasterizing glyphs once and reusing them anymore than arbitrary vector graphics renderers can. I sure hope my vector graphics editors cache the rasterizations instead of re-rasterizing needlessly.

rayiner 4y ago

But you don’t have to do it every frame. Most apps could just render to bitmaps once and cache the results.

userbinator 4y ago

You shouldn't even need a GPU for basic UI like this, especially with "modern" ultra-minimalist flat UIs.

vladvasiliu 4y ago
Yeah, but they're not actually completely flat since they're translucent. Also, on Windows, the "acrylic" thing is a circus show. Every app behaves differently. It also manages to sometimes lag on an RX5600. I know it's an "old" gpu, but still...

jcelerier 4y ago

Maybe basic productivity applications shouldn't try to tackle open research problems and instead just do the thing which has worked fine for the last 30 years which is CPU rendering? Like, do you think notepad is going to burn your battery because it redraws 20x20 pixels on every mouse event ?

moonchild 4y ago

It is being worked on (ahem...) But in most contexts, it is not necessary to render vector graphics 'live'; caching is acceptable, so this should not be a bottleneck for most desktop software.