4 ms·

I am really interested in your research related to vector graphics!

slimsag 5y ago
It'll be a bit before I can publish the paper, libraries, tooling etc. around this - it's been maybe 8 years in the making but slow - the general idea is: 1. GPUs are actually _ridiculously and stupidly_ excellent at natively rendering Bézier curves if you "limit" yourself to quadratic curves. 2. Most of the reasons designers / end users hate working with Bézier curves is because of the absolutely terrible control point manipulation that cubic curves require. 3. GPUs cannot natively render cubic curves (look at any of the papers on this and you'll find they do CPU-side translation of cubic -> quadratic before sending to the GPU in the best case) which harms perf substantially and makes animation of curves expensive. Plus handling the intersection rules is incredibly complex. ...so maybe using the 'mathematically pure' cubic curves is not actually so great after all. :) if only we had tools and formats (not SVG) that worked on isolated quadratic curves (without care for intersection) - then GPU rendering implementations wouldn't need to be absolutely horridly complex (nvidia et. al), or terrible approximations (valve SDF), we could literally send a vector graphic model directly to the GPU and have it render (and animate!) at incredible speeds, with easier to understand tools. At least - that's the idea.
- Jarred 5y ago
  What do you think of NURBS? eg http://verbnurbs.com/ http://verbnurbs.com/. It’s intended for 3D instead of 2D, but still kind of relevant How do you plan on doing text rendering? I feel like that’s often a challenge with cross-platform UI frameworks. Electron/Chromium does this well. Often QT apps don’t handle pixel ratio correctly, or the text looks non-native.
  - pcwalton 5y ago
    NURBS are a generalization of Bézier curves, so everything that applies to Bézier curves here applies to NURBS as well.
    - wnkrshm 5y ago
      There are applications in CAD where I imagine working with degree 2 surfaces/curves only is going to be very tricky. What about differentials? How do you get a change of normal, how do you get a curvature tensor? Without a continuous 2nd derivative? Edit: In e.g. product design like e.g. cars, people want curvature continuity, i.e. C2 at minimum for their surfaces, because on machined surfaces, you see any curvature discontinuities as edges in reflected light... that's why degree 3 is kind of an industry standard.
- Jasper_ 5y ago
  I wouldn't strictly say that. Even with quadratic curves, you still have a serial process of parsing a shape to determine interior vs. exterior. This is true of any non-convex polygon, not just those with curves. You have to solve that sort somehow, whether it be through linear search (e.g. Slug), through geometry simplification (e.g. Loop-Blinn, triangulation), through signed area tricks (e.g. red book, style stencil-and-cover), or through brute force (e.g. SDFs). Eliminating cubics eliminates some trickier problems, but I don't think it fully solves the issues.
  - slimsag 5y ago
    Yep, you probably read my comment before I edited and clarified, but you also need to eliminate overlapping geometry. Basically just build your graphics around non-overlapping convex and concave quadratic curves represented as triangles. The trick is in building tooling that makes that a pleasant experience (and do any required reductions of more complex shapes -> isolated quadratic curves at tool time, not at render time.) Existing approaches try to fit what we have (SVGs, complex polygons) onto what GPUs can do. This approach tries to take what GPUs _can do_ and make that competitive with what we have (SVGs, etc.)
    - megameter 5y ago
      In working with existing graphics tools to make drawings, something I've come to realize is that overlap is crucial to the workflow: it's easier to draw the line you want by overshooting and erasing than it is to get it precisely drafted. Likewise it's much easier to paint within a stencil mask than it is to stay within the lines freehand. To me, that suggests the main distinction between source and output formats comes from how/whether they go about decimating the "erasure" data, since the final result produces much more complex non-overlapped paths. The attractiveness of the Bezier usually isn't in the initial definition of a shape - it just happens to be great at reproduction of multiple curves cut and spliced together. So it has the feel of a premature optimization that probably should be reconsidered in making more ergonomic tooling.
- pcwalton 5y ago
  > 3. GPUs cannot natively render cubic curves (look at any of the papers on this and you'll find they do CPU-side translation of cubic -> quadratic before sending to the GPU in the best case) which harms perf substantially and makes animation of curves expensive. Plus handling the intersection rules is incredibly complex. Loop-Blinn natively renders cubic curves in the same way as quadratic curves (implicitization), with some preprocessing to classify cubic curves based on number of inflection points (but this is not the same as converting to quadratic). But I don't think the complexity is really worth it when you can just approximate cubic Béziers with quadratic Béziers. > we could literally send a vector graphic model directly to the GPU and have it render (and animate!) at incredible speeds, with easier to understand tools. You can still submit high-level display lists to the GPU with an all-quadratic pipeline. Just add a compute shader step that runs before your pipeline that converts your high-level vector display lists to whatever primitives are easiest for you to render. You may want to look at Raph Levien's piet-gpu, which implements a lot of this idea. Google's Spinel is also a good resource, though the code is impenetrable.
  - slimsag 5y ago
    My understand is that Loop-Blinn performs constrained Delaunay triangulation prior to uploading an actual display list to the GPU, in effect turning its cubic curves into quadratic ones, from the paper: > The overlap removal and triangulation is a one-time preprocess that takes place on the CPU All I'm suggesting is merely to ditch that process, ditch the idea that you can have intersecting lines, and instead design tools around convex and concave isolated triangles[0] (quadratic curves) so you end up composing shapes like the ones from the Loop-Blinn paper[1] directly, with your tools aiding you in composing them rather than more complex shapes (cubic curves, intersecting segments, etc.) being in the actual model one needs to render. I don't think any of this is truly novel from a research POV or anything, it's just a different approach that I think can in practice lead to far better real-world application. It's considering the whole picture from designer -> application instead of the more limited (and much harder) "start with SVGs, how do we render them on GPUs?" question. [0] https://imgur.com/a/ferSScN https://imgur.com/a/ferSScN [1] https://imgur.com/a/VHZ8Ers https://imgur.com/a/VHZ8Ers

It'll be a bit before I can publish the paper, libraries, tooling etc. around this - it's been maybe 8 years in the making but slow - the general idea is: 1. GPUs are actually _ridiculously and stupidly_ excellent at natively rendering Bézier curves if you "limit" yourself to quadratic curves. 2. Most of the reasons designers / end users hate working with Bézier curves is because of the absolutely terrible control point manipulation that cubic curves require. 3. GPUs cannot natively render cubic curves (look at any of the papers on this and you'll find they do CPU-side translation of cubic -> quadratic before sending to the GPU in the best case) which harms perf substantially and makes animation of curves expensive. Plus handling the intersection rules is incredibly complex. ...so maybe using the 'mathematically pure' cubic curves is not actually so great after all. :) if only we had tools and formats (not SVG) that worked on isolated quadratic curves (without care for intersection) - then GPU rendering implementations wouldn't need to be absolutely horridly complex (nvidia et. al), or terrible approximations (valve SDF), we could literally send a vector graphic model directly to the GPU and have it render (and animate!) at incredible speeds, with easier to understand tools. At least - that's the idea.

Jarred 5y ago
What do you think of NURBS? eg http://verbnurbs.com/ http://verbnurbs.com/. It’s intended for 3D instead of 2D, but still kind of relevant How do you plan on doing text rendering? I feel like that’s often a challenge with cross-platform UI frameworks. Electron/Chromium does this well. Often QT apps don’t handle pixel ratio correctly, or the text looks non-native.
- pcwalton 5y ago
  NURBS are a generalization of Bézier curves, so everything that applies to Bézier curves here applies to NURBS as well.
  - wnkrshm 5y ago
    There are applications in CAD where I imagine working with degree 2 surfaces/curves only is going to be very tricky. What about differentials? How do you get a change of normal, how do you get a curvature tensor? Without a continuous 2nd derivative? Edit: In e.g. product design like e.g. cars, people want curvature continuity, i.e. C2 at minimum for their surfaces, because on machined surfaces, you see any curvature discontinuities as edges in reflected light... that's why degree 3 is kind of an industry standard.
Jasper_ 5y ago
I wouldn't strictly say that. Even with quadratic curves, you still have a serial process of parsing a shape to determine interior vs. exterior. This is true of any non-convex polygon, not just those with curves. You have to solve that sort somehow, whether it be through linear search (e.g. Slug), through geometry simplification (e.g. Loop-Blinn, triangulation), through signed area tricks (e.g. red book, style stencil-and-cover), or through brute force (e.g. SDFs). Eliminating cubics eliminates some trickier problems, but I don't think it fully solves the issues.
- slimsag 5y ago
  Yep, you probably read my comment before I edited and clarified, but you also need to eliminate overlapping geometry. Basically just build your graphics around non-overlapping convex and concave quadratic curves represented as triangles. The trick is in building tooling that makes that a pleasant experience (and do any required reductions of more complex shapes -> isolated quadratic curves at tool time, not at render time.) Existing approaches try to fit what we have (SVGs, complex polygons) onto what GPUs can do. This approach tries to take what GPUs _can do_ and make that competitive with what we have (SVGs, etc.)
  - megameter 5y ago
    In working with existing graphics tools to make drawings, something I've come to realize is that overlap is crucial to the workflow: it's easier to draw the line you want by overshooting and erasing than it is to get it precisely drafted. Likewise it's much easier to paint within a stencil mask than it is to stay within the lines freehand. To me, that suggests the main distinction between source and output formats comes from how/whether they go about decimating the "erasure" data, since the final result produces much more complex non-overlapped paths. The attractiveness of the Bezier usually isn't in the initial definition of a shape - it just happens to be great at reproduction of multiple curves cut and spliced together. So it has the feel of a premature optimization that probably should be reconsidered in making more ergonomic tooling.
pcwalton 5y ago
> 3. GPUs cannot natively render cubic curves (look at any of the papers on this and you'll find they do CPU-side translation of cubic -> quadratic before sending to the GPU in the best case) which harms perf substantially and makes animation of curves expensive. Plus handling the intersection rules is incredibly complex. Loop-Blinn natively renders cubic curves in the same way as quadratic curves (implicitization), with some preprocessing to classify cubic curves based on number of inflection points (but this is not the same as converting to quadratic). But I don't think the complexity is really worth it when you can just approximate cubic Béziers with quadratic Béziers. > we could literally send a vector graphic model directly to the GPU and have it render (and animate!) at incredible speeds, with easier to understand tools. You can still submit high-level display lists to the GPU with an all-quadratic pipeline. Just add a compute shader step that runs before your pipeline that converts your high-level vector display lists to whatever primitives are easiest for you to render. You may want to look at Raph Levien's piet-gpu, which implements a lot of this idea. Google's Spinel is also a good resource, though the code is impenetrable.
- slimsag 5y ago
  My understand is that Loop-Blinn performs constrained Delaunay triangulation prior to uploading an actual display list to the GPU, in effect turning its cubic curves into quadratic ones, from the paper: > The overlap removal and triangulation is a one-time preprocess that takes place on the CPU All I'm suggesting is merely to ditch that process, ditch the idea that you can have intersecting lines, and instead design tools around convex and concave isolated triangles[0] (quadratic curves) so you end up composing shapes like the ones from the Loop-Blinn paper[1] directly, with your tools aiding you in composing them rather than more complex shapes (cubic curves, intersecting segments, etc.) being in the actual model one needs to render. I don't think any of this is truly novel from a research POV or anything, it's just a different approach that I think can in practice lead to far better real-world application. It's considering the whole picture from designer -> application instead of the more limited (and much harder) "start with SVGs, how do we render them on GPUs?" question. [0] https://imgur.com/a/ferSScN https://imgur.com/a/ferSScN [1] https://imgur.com/a/VHZ8Ers https://imgur.com/a/VHZ8Ers