6 ms·

I don't want to be the guy that doesn't read the entire article, but the first sentence surprised me quite a bit: > Despite vector graphics being used in every

I don't want to be the guy that doesn't read the entire article, but the first sentence surprised me quite a bit: > Despite vector graphics being used in every computer with a screen connected to it, rendering of vector shapes and text is still mostly a task for the CPU. Do modern vector libraries really not use the GPU? One of the very first things I did when learning Vulkan was to use a fragment shader to draw a circle inside a square polygon. I always assumed that we've been using the GPU for pretty much any sort of vector rasterization, whether it was bezier curves or font rendering.

amelius 4y ago
Yes, it's like the article is trying to ignore why GPUs were invented in the first place.
TazeTSchnitzel 4y ago
There's definitely a lot of code out there that still does this only on the CPU, but the optimized implementations used in modern OSes, browsers and games won't.
ygra 4y ago
A few of the previous approaches are mentioned in Other work near the end. And from reading a few articles on the topic I got the impression that, yes, drawing a single shape in a shader seems almost trivial, vector graphics in general means mostly what PostScript/PDF/SVG are capable of these days. This means you don't just need filled shapes, but also strokes (and stroking in itself is a quite complicated problem), including dashed lines, line caps, etc. Gradients, image fills, blending modes are probably on the more trivial end, since I think those can all be easily solved in shaders.
- bXVsbGVy 4y ago
  In SVG, strokes has a separated specification only for them [1]. The specification has images that highlights some of the challenges. [1] https://svgwg.org/specs/strokes/ https://svgwg.org/specs/strokes/
Guzba 4y ago
SVG paths can be arbitrarily complex. This article really doesn't discuss any of the actual hard cases. For example, imagine the character S rotated 1 degree and added to the path on top of itself in a full rotation. This is one path composed of 360 shapes. These begin and end fill sections (winding order changes) coincide in the same pixels at arbitrary angles (and the order of the hits is not automatically sorted!) but the final color cannot be arrived at correctly if you do not process all of the shapes at the same time. If you do them one at a time, you'll blend tiny (perhaps rounded to zero) bits of color and end up with a mess that looks nothing like what it should. These are often called conflation artifacts IIRC. There's way more to this than drawing circles and rectangles, and these hard cases are why much of path / vector graphics filling still ends up being better on CPU where you can accumulate, sort, etc which takes a lot of the work away. CPU does basically per-Y whereas this is GPU per-pixel so perhaps they're almost equal if the GPU has the square of a CPU power. Obv this isn't quite right but gives you an idea. Video discussing path filling on CPU (super sampling and trapezoid): https://youtu.be/Did21OYIrGI?t=318 https://youtu.be/Did21OYIrGI?t=318 We don't talk about the complex cases but this at least may help explain the simple stuff on CPU for those curious.
- Scene_Cast2 4y ago
  Is is it practical to ignore / approximate / offload the complex edge cases?
  - Guzba 4y ago
    I want to say yes, but it depends on what you're actually doing as a final goal. Detecting when a path is antagonistic to most GPU approaches takes time, as does preparing the data however it needs to be prepared on the CPU before being uploaded to the GPU. If you can just fill the whole thing on CPU in that time, you wasted your time even thinking about the GPU. If you can identify a simple case quickly, it's probably totally a good idea to get the path done on the GPU unless you need to bring the pixels back to the CPU, maybe for writing to disk. The upload and then download can be way slower than just, again, filling on CPU. If you're filling on GPU and then using on GPU (maybe as a web renderer or something), GPU is probably a big win. Except, this may not actually matter. If there is no need to re-render the path after the first time, it would be dumb to keep re-rendering on the GPU each frame / screen paint. Instead you'd want to put it into a texture. Well.... if you're only rendering once and putting into a texture, this whole conversation is maybe pointless? Then what is simple is probably the best idea. Anyway lots to 2d graphics that goes underappreciated!
dragontamer 4y ago
3D vector graphics are not as full featured as 2d vector graphics. 2d vector graphics include things like "bones" and "tweening", which are CPU algorithms. (Much like how bone processing in 3d world is also CPU-side processing). --------- Consider the creation of a Beizer curve, in 2d or 3d. Do you expect this to be a CPU algorithm, or GPU algorithm? Answer: clearly a CPU algorithm. GPU algorithms generally are triangle-only, or close to it (ex: quads) as far as geometry. Sure, there are geometry shaders, but I don't think its common practice to take a Beizer Curve definition and write a Tesselator-shader for it and output (in parallel) a set of verticies. (And if someone is doing that, I'm interested in heading / learning more about it. It seems like a parallelizable algorithm to me but the devil is always in the details...).
- jstimpfle 4y ago
  GPUs have evolved away from being strictly triangle rasterizers. There are compute shaders that can do general purpose computing. The approach described here could in theory be set up by "drawing" a single quad - the whole screen, and it doesn't even need compute shaders but can be implemented using conventional vertex/fragment shaders with global buffer access (in OpenGL, UBOs or SSBOs). There is a well-known paper that describes an approach how to draw bezier curves by "drawing" a single triangle. Checkout Loop-Blinn from 2005.
  - Lichtso 4y ago
    Ah yes, the https://en.wikipedia.org/wiki/Implicit_curve https://en.wikipedia.org/wiki/Implicit_curve approach to filling curves. I have implemented a GPU vector renderer using that: https://github.com/Lichtso/contrast_renderer https://github.com/Lichtso/contrast_renderer Here the implicit curve filling extracted as a shader toy: https://www.shadertoy.com/view/WlcBRn https://www.shadertoy.com/view/WlcBRn It can even do cubic rational bezier curves, resolution independently. And to my knowledge it is the only renderer capable of that so far.
    - jstimpfle 4y ago
      Your project sounds very impressive. I would like to try it out, unfortunately I'm unlikely to be able to get it to run by building Rust. If I understand correctly it should be able to run it on the Web, you have a demo somewhere? Or a video?
Jasper_ 4y ago
Skia mostly uses the CPU -- it can draw some very basic stuff on the GPU, but text and curves are a CPU fallback. Quartz 2D is full CPU. cairo never got an acceptable GPU path. Direct2D is the tessellate-to-triangle approach. If you name a random vector graphics library, chances are 99% of the time it will be using the CPU.
- jahewson 4y ago
  Skia can tessellate curved paths.
  - pcwalton 4y ago
    Skia has code paths for everything: CPU path drawing, CPU tessellation followed by GPU rasterization with special paths for convex vs. concave paths, NV_path_rendering, Spinel/Skia Compute... It's actually hard to figure out what it's doing because it depends so much on the particular configuration.

amelius 4y ago

Yes, it's like the article is trying to ignore why GPUs were invented in the first place.

TazeTSchnitzel 4y ago

There's definitely a lot of code out there that still does this only on the CPU, but the optimized implementations used in modern OSes, browsers and games won't.

ygra 4y ago

A few of the previous approaches are mentioned in Other work near the end. And from reading a few articles on the topic I got the impression that, yes, drawing a single shape in a shader seems almost trivial, vector graphics in general means mostly what PostScript/PDF/SVG are capable of these days. This means you don't just need filled shapes, but also strokes (and stroking in itself is a quite complicated problem), including dashed lines, line caps, etc. Gradients, image fills, blending modes are probably on the more trivial end, since I think those can all be easily solved in shaders.

bXVsbGVy 4y ago
In SVG, strokes has a separated specification only for them [1]. The specification has images that highlights some of the challenges. [1] https://svgwg.org/specs/strokes/ https://svgwg.org/specs/strokes/

Guzba 4y ago

SVG paths can be arbitrarily complex. This article really doesn't discuss any of the actual hard cases. For example, imagine the character S rotated 1 degree and added to the path on top of itself in a full rotation. This is one path composed of 360 shapes. These begin and end fill sections (winding order changes) coincide in the same pixels at arbitrary angles (and the order of the hits is not automatically sorted!) but the final color cannot be arrived at correctly if you do not process all of the shapes at the same time. If you do them one at a time, you'll blend tiny (perhaps rounded to zero) bits of color and end up with a mess that looks nothing like what it should. These are often called conflation artifacts IIRC. There's way more to this than drawing circles and rectangles, and these hard cases are why much of path / vector graphics filling still ends up being better on CPU where you can accumulate, sort, etc which takes a lot of the work away. CPU does basically per-Y whereas this is GPU per-pixel so perhaps they're almost equal if the GPU has the square of a CPU power. Obv this isn't quite right but gives you an idea. Video discussing path filling on CPU (super sampling and trapezoid): https://youtu.be/Did21OYIrGI?t=318 https://youtu.be/Did21OYIrGI?t=318 We don't talk about the complex cases but this at least may help explain the simple stuff on CPU for those curious.

Scene_Cast2 4y ago
Is is it practical to ignore / approximate / offload the complex edge cases?
- Guzba 4y ago
  I want to say yes, but it depends on what you're actually doing as a final goal. Detecting when a path is antagonistic to most GPU approaches takes time, as does preparing the data however it needs to be prepared on the CPU before being uploaded to the GPU. If you can just fill the whole thing on CPU in that time, you wasted your time even thinking about the GPU. If you can identify a simple case quickly, it's probably totally a good idea to get the path done on the GPU unless you need to bring the pixels back to the CPU, maybe for writing to disk. The upload and then download can be way slower than just, again, filling on CPU. If you're filling on GPU and then using on GPU (maybe as a web renderer or something), GPU is probably a big win. Except, this may not actually matter. If there is no need to re-render the path after the first time, it would be dumb to keep re-rendering on the GPU each frame / screen paint. Instead you'd want to put it into a texture. Well.... if you're only rendering once and putting into a texture, this whole conversation is maybe pointless? Then what is simple is probably the best idea. Anyway lots to 2d graphics that goes underappreciated!

dragontamer 4y ago

3D vector graphics are not as full featured as 2d vector graphics. 2d vector graphics include things like "bones" and "tweening", which are CPU algorithms. (Much like how bone processing in 3d world is also CPU-side processing). --------- Consider the creation of a Beizer curve, in 2d or 3d. Do you expect this to be a CPU algorithm, or GPU algorithm? Answer: clearly a CPU algorithm. GPU algorithms generally are triangle-only, or close to it (ex: quads) as far as geometry. Sure, there are geometry shaders, but I don't think its common practice to take a Beizer Curve definition and write a Tesselator-shader for it and output (in parallel) a set of verticies. (And if someone is doing that, I'm interested in heading / learning more about it. It seems like a parallelizable algorithm to me but the devil is always in the details...).

jstimpfle 4y ago
GPUs have evolved away from being strictly triangle rasterizers. There are compute shaders that can do general purpose computing. The approach described here could in theory be set up by "drawing" a single quad - the whole screen, and it doesn't even need compute shaders but can be implemented using conventional vertex/fragment shaders with global buffer access (in OpenGL, UBOs or SSBOs). There is a well-known paper that describes an approach how to draw bezier curves by "drawing" a single triangle. Checkout Loop-Blinn from 2005.
- Lichtso 4y ago
  Ah yes, the https://en.wikipedia.org/wiki/Implicit_curve https://en.wikipedia.org/wiki/Implicit_curve approach to filling curves. I have implemented a GPU vector renderer using that: https://github.com/Lichtso/contrast_renderer https://github.com/Lichtso/contrast_renderer Here the implicit curve filling extracted as a shader toy: https://www.shadertoy.com/view/WlcBRn https://www.shadertoy.com/view/WlcBRn It can even do cubic rational bezier curves, resolution independently. And to my knowledge it is the only renderer capable of that so far.
  - jstimpfle 4y ago
    Your project sounds very impressive. I would like to try it out, unfortunately I'm unlikely to be able to get it to run by building Rust. If I understand correctly it should be able to run it on the Web, you have a demo somewhere? Or a video?

Jasper_ 4y ago

Skia mostly uses the CPU -- it can draw some very basic stuff on the GPU, but text and curves are a CPU fallback. Quartz 2D is full CPU. cairo never got an acceptable GPU path. Direct2D is the tessellate-to-triangle approach. If you name a random vector graphics library, chances are 99% of the time it will be using the CPU.

jahewson 4y ago
Skia can tessellate curved paths.
- pcwalton 4y ago
  Skia has code paths for everything: CPU path drawing, CPU tessellation followed by GPU rasterization with special paths for convex vs. concave paths, NV_path_rendering, Spinel/Skia Compute... It's actually hard to figure out what it's doing because it depends so much on the particular configuration.