5 ms·

It would be fantastic if something like this were part of the modern apis. Vulkan, Metal DX12. But I guess it's not as sexy as raytracing.

moonchild 6y ago
Nvidia tried to make it happen[0]. Sadly, it didn't catch on. 0. https://developer.nvidia.com/nv-path-rendering https://developer.nvidia.com/nv-path-rendering
- slimsag 6y ago
  It's still there on all NVIDIA GPUs as an extension, just nobody uses it. IMO it didn't catch on because all three of these points: 1. It only works on NVIDIA GPUs, and is riddled with joint patents from NVIDIA and Microsoft forbidding anyone like AMD or Intel from supporting it. 2. It's hard to use: you need to get your data into a format it can consume, usage is non-trivial, and often video game artists are already working with rasterized textures anyway so it's easy to omit. 3. Vector graphics suck for artists. The number of graphic designers I have met (who are the most likely subset of artists to work with vector graphics) that simply hate or do not understand the cubic bezier curve control points in Adobe, Inkscape, and other tools is incredible.
  - derefr 6y ago
    > It only works on NVIDIA GPUs, and is riddled with joint patents from NVIDIA and Microsoft forbidding anyone like AMD or Intel from supporting it. Why do companies do this? What do they expect to get out of creating an API that is proprietary to their specific non-monopoly device, and that therefore very obviously nobody will ever actually use?
    - numlock86 6y ago
      Well, you could always implement a fallback for unsupported devices and advertise for better performance on supported devices. Implementation will usually be "sponsored" by the one providing the technology. If the feature then gets adopted enough others will have to come up with something similar or support it native, too. This happens all the time with GPU and CPU vendors, both in gaming and industry.
      - derefr 6y ago
        Maybe, but I got the sense from the article that what NVIDIA has patented are the semantics of the API itself (i.e. the types of messages sent between the CPU and GPU.) A polyfill for the API might still be infringing, given that it would need to expose the same datatypes on the CPU side. And even if it wasn’t, the path-shader semantics are so different from those of regular 2D UI frameworks, that a 2D UI framework implemented to use path-shading, that falls back to using the polyfill, might be much worse performing than one implemented using regular CPU-side 2D plotting. It would very likely also suffer from issues like flicker, judder, etc, which are much worse/more noticable than just “increased CPU usage”.
    - adwn 6y ago
      > What do they expect to get out of creating an API that is proprietary to their specific non-monopoly device, and that therefore very obviously nobody will ever actually use? You mean like Cuda, which is wildly successful, has a huge ecosystem, and which basically ensures you'll have to buy NVidia if you're serious about GPU computing?
      - derefr 6y ago
        CUDA is, essentially, a B2B product-feature. A business is flexible when serving its own needs: if they want to do GPGPU computations, they can evaluate the alternatives and choose a platform/technology like CUDA to build on; and so choose to lock themselves into the particular GPUs that implement that technology. But that’s a choice they’re only making for themselves. They’re using those GPUs to compute something; they’re not forcing anyone else downstream of them to use those same GPUs to consume what they produce. The GPUs they’re using become part of their black box. Path shaders, on the other hand, would—at least as described in the article—be chiefly a B2C / end-user product-feature. They’d be something games/apps would rely on to draw 2D UI elements. But path shaders would need to be implemented† by the developers creating those games/apps. † (Yes, really, they would need to be implemented by the game/app developer, not game-engine framework devs. Path shaders are a low-level optimization, like Vulkan — something that is only advantageous to use when you use it directly to take advantage of its low-level semantics, rather than through a framework that tries to wrap it in some other, more traditional semantics. As with OpenGL, the ill-fitted abstractions of the traditional API were precisely what made it slow!) And those game/app developers, unlike the developers doing GPGPU computations, have to make a decision based on what devices their customers have, rather than what their own company can buy. (Or, if they think they have a “killer app”, they can try to force their customers to buy into the platform they built on. But most companies who think they’ve built a B2C killer app, haven’t.) For a B2B product-feature, a plurality position is fine. Even one forced to be permanent by patents. For a B2C product-feature, a plurality position by itself is fine, because rivals will just ship something similar and offer cross-support. But an indefinite plurality position is death. Compare, in the web ecosystem: • Features shipped originally in one renderer as “clean, standalone” ideas, e.g. -webkit-border-radius. These were cloned (-moz-border-radius, etc.) and then standardized (plain old border-radius.) • Features shipped in one renderer as implementations dependent on that renderer’s particular environment/ecosystem, e.g. IE’s ActiveX-based CSS “filter” stanza. Because of how they were implemented, these had no potential to ever be copied by rivals, so nobody outside of Microsoft bothered to use them on their sites, since they knew that only IE would render the results correctly, and IE at that point already had only a plurality position.
    - d110af5ccf 6y ago
      > What do they expect to get out of ... Nvidia did exactly this with CUDA, so go take a look at the ML world for an example of how it works. It seems to be going quite well for them. A common enough refrain is "I don't really want to buy Nvidia, but my toolchain requires CUDA". Pretty much every FPGA and SoC vendor does exactly this as far as I understand things. It's why you can't trivially run an open OS on most mobile hardware. Apparently such schemes don't meaningfully affect the purchasing decisions of a sufficiently large fraction of people to disincentivize the behavior.
      - my123 6y ago
        Intel and AMD also did the exact same thing with x86. But yet, people still use it. (and Arm also has its fair share of patents, but is a licensable architecture by others)
    - my123 6y ago
      Intel and NVIDIA have a patent cross-licensing agreement at the time, which is still ongoing for all patents filed on or prior to March 31, 2017. https://www.sec.gov/Archives/edgar/data/1045810/000119312511005134/dex101.htm https://www.sec.gov/Archives/edgar/data/1045810/000119312511... Patents here are used for mutual destruction in case one tries to sue the other, NVIDIA won't use them against AMD (and the reverse is true too) when regarding GPUs. Intel and AMD would however enforce their patents if NVIDIA tries to enter the x86 CPU market.
  - redisman 6y ago
    > Vector graphics suck for artists Someone mentioned Flash in this thread and that was a very approachable vector graphics tool. I don't know how many games translate to the vector style though - it's almost invariably a cartoonish look. The tools are very geometric so it just kind of nudges you towards that. Pixels these days are more like painting so it's no surprise artists like that workflow (they all secretly want to be painting painters).
    - monsieurbanana 6y ago
      > they all secretly want to be painting painters This is going to be literally life changing for them! Quick, someone inform the artists of that!
  - Const-me 6y ago
    > just nobody uses it AFAIK Skia and derived works (Chrome, Chromium, Electron, etc.), are all using it when available.
- Jasper_ 6y ago
  Stencil-and-cover approaches like NV_path_rendering have a lot of drawbacks like documented below, but probably biggest of all, they're still mostly just doing the tessellation on CPU. A lot of the important things, like winding mode calculations, are handled on the CPU. Modern research is looking for ways out of that.
  - Lichtso 6y ago
    Actually, they calculate the winding per fragment on the GPU [0]. They require polygon tessellation only for stroking (which has no winding rule). The downside of their approach is that it is memory bandwidth limited, precisely because it does the winding on the GPU instead of using CPU tessellation to avoid overlap / overdraw. Curve filling is pretty much solved with implicit curves, stencil-and-cover or scanline-intersection-sort approaches (all of which can be done in a GPU only fashion). Stroking is where things could still improve a lot as it is almost always approximated by polygons. [0]: https://developer.nvidia.com/gpu-accelerated-path-rendering https://developer.nvidia.com/gpu-accelerated-path-rendering
raphlinus 6y ago
I think the world is going into a more layered approach, where it's the job of the driver API (Vulkan etc) to expose the power of the hardware fairly directly, and it's the job of higher levels to express the actual rendering in terms of those lower level primitives. Raytracing is an exception because you do need hardware support to do it well. Whether there's hardware that can make 2D rendering work better is an intriguing question. The NV path rendering stuff (mentioned elsethread) was an attempt (though I think it may be more driver/API than hardware), but I believe my research direction is better, in that it will be higher quality, faster, and more flexible with respect to the imaging model on standard compute shaders than an approach using the NV path rendering primitives. Obviously I have to back that up with empirical measurements, which is not yet done :)

moonchild 6y ago

Nvidia tried to make it happen[0]. Sadly, it didn't catch on. 0. https://developer.nvidia.com/nv-path-rendering https://developer.nvidia.com/nv-path-rendering

slimsag 6y ago
It's still there on all NVIDIA GPUs as an extension, just nobody uses it. IMO it didn't catch on because all three of these points: 1. It only works on NVIDIA GPUs, and is riddled with joint patents from NVIDIA and Microsoft forbidding anyone like AMD or Intel from supporting it. 2. It's hard to use: you need to get your data into a format it can consume, usage is non-trivial, and often video game artists are already working with rasterized textures anyway so it's easy to omit. 3. Vector graphics suck for artists. The number of graphic designers I have met (who are the most likely subset of artists to work with vector graphics) that simply hate or do not understand the cubic bezier curve control points in Adobe, Inkscape, and other tools is incredible.
- derefr 6y ago
  > It only works on NVIDIA GPUs, and is riddled with joint patents from NVIDIA and Microsoft forbidding anyone like AMD or Intel from supporting it. Why do companies do this? What do they expect to get out of creating an API that is proprietary to their specific non-monopoly device, and that therefore very obviously nobody will ever actually use?
  - numlock86 6y ago
    Well, you could always implement a fallback for unsupported devices and advertise for better performance on supported devices. Implementation will usually be "sponsored" by the one providing the technology. If the feature then gets adopted enough others will have to come up with something similar or support it native, too. This happens all the time with GPU and CPU vendors, both in gaming and industry.
    - derefr 6y ago
      Maybe, but I got the sense from the article that what NVIDIA has patented are the semantics of the API itself (i.e. the types of messages sent between the CPU and GPU.) A polyfill for the API might still be infringing, given that it would need to expose the same datatypes on the CPU side. And even if it wasn’t, the path-shader semantics are so different from those of regular 2D UI frameworks, that a 2D UI framework implemented to use path-shading, that falls back to using the polyfill, might be much worse performing than one implemented using regular CPU-side 2D plotting. It would very likely also suffer from issues like flicker, judder, etc, which are much worse/more noticable than just “increased CPU usage”.
  - adwn 6y ago
    > What do they expect to get out of creating an API that is proprietary to their specific non-monopoly device, and that therefore very obviously nobody will ever actually use? You mean like Cuda, which is wildly successful, has a huge ecosystem, and which basically ensures you'll have to buy NVidia if you're serious about GPU computing?
    - derefr 6y ago
      CUDA is, essentially, a B2B product-feature. A business is flexible when serving its own needs: if they want to do GPGPU computations, they can evaluate the alternatives and choose a platform/technology like CUDA to build on; and so choose to lock themselves into the particular GPUs that implement that technology. But that’s a choice they’re only making for themselves. They’re using those GPUs to compute something; they’re not forcing anyone else downstream of them to use those same GPUs to consume what they produce. The GPUs they’re using become part of their black box. Path shaders, on the other hand, would—at least as described in the article—be chiefly a B2C / end-user product-feature. They’d be something games/apps would rely on to draw 2D UI elements. But path shaders would need to be implemented† by the developers creating those games/apps. † (Yes, really, they would need to be implemented by the game/app developer, not game-engine framework devs. Path shaders are a low-level optimization, like Vulkan — something that is only advantageous to use when you use it directly to take advantage of its low-level semantics, rather than through a framework that tries to wrap it in some other, more traditional semantics. As with OpenGL, the ill-fitted abstractions of the traditional API were precisely what made it slow!) And those game/app developers, unlike the developers doing GPGPU computations, have to make a decision based on what devices their customers have, rather than what their own company can buy. (Or, if they think they have a “killer app”, they can try to force their customers to buy into the platform they built on. But most companies who think they’ve built a B2C killer app, haven’t.) For a B2B product-feature, a plurality position is fine. Even one forced to be permanent by patents. For a B2C product-feature, a plurality position by itself is fine, because rivals will just ship something similar and offer cross-support. But an indefinite plurality position is death. Compare, in the web ecosystem: • Features shipped originally in one renderer as “clean, standalone” ideas, e.g. -webkit-border-radius. These were cloned (-moz-border-radius, etc.) and then standardized (plain old border-radius.) • Features shipped in one renderer as implementations dependent on that renderer’s particular environment/ecosystem, e.g. IE’s ActiveX-based CSS “filter” stanza. Because of how they were implemented, these had no potential to ever be copied by rivals, so nobody outside of Microsoft bothered to use them on their sites, since they knew that only IE would render the results correctly, and IE at that point already had only a plurality position.
  - d110af5ccf 6y ago
    > What do they expect to get out of ... Nvidia did exactly this with CUDA, so go take a look at the ML world for an example of how it works. It seems to be going quite well for them. A common enough refrain is "I don't really want to buy Nvidia, but my toolchain requires CUDA". Pretty much every FPGA and SoC vendor does exactly this as far as I understand things. It's why you can't trivially run an open OS on most mobile hardware. Apparently such schemes don't meaningfully affect the purchasing decisions of a sufficiently large fraction of people to disincentivize the behavior.
    - my123 6y ago
      Intel and AMD also did the exact same thing with x86. But yet, people still use it. (and Arm also has its fair share of patents, but is a licensable architecture by others)
  - my123 6y ago
    Intel and NVIDIA have a patent cross-licensing agreement at the time, which is still ongoing for all patents filed on or prior to March 31, 2017. https://www.sec.gov/Archives/edgar/data/1045810/000119312511005134/dex101.htm https://www.sec.gov/Archives/edgar/data/1045810/000119312511... Patents here are used for mutual destruction in case one tries to sue the other, NVIDIA won't use them against AMD (and the reverse is true too) when regarding GPUs. Intel and AMD would however enforce their patents if NVIDIA tries to enter the x86 CPU market.
- redisman 6y ago
  > Vector graphics suck for artists Someone mentioned Flash in this thread and that was a very approachable vector graphics tool. I don't know how many games translate to the vector style though - it's almost invariably a cartoonish look. The tools are very geometric so it just kind of nudges you towards that. Pixels these days are more like painting so it's no surprise artists like that workflow (they all secretly want to be painting painters).
  - monsieurbanana 6y ago
    > they all secretly want to be painting painters This is going to be literally life changing for them! Quick, someone inform the artists of that!
- Const-me 6y ago
  > just nobody uses it AFAIK Skia and derived works (Chrome, Chromium, Electron, etc.), are all using it when available.
Jasper_ 6y ago
Stencil-and-cover approaches like NV_path_rendering have a lot of drawbacks like documented below, but probably biggest of all, they're still mostly just doing the tessellation on CPU. A lot of the important things, like winding mode calculations, are handled on the CPU. Modern research is looking for ways out of that.
- Lichtso 6y ago
  Actually, they calculate the winding per fragment on the GPU [0]. They require polygon tessellation only for stroking (which has no winding rule). The downside of their approach is that it is memory bandwidth limited, precisely because it does the winding on the GPU instead of using CPU tessellation to avoid overlap / overdraw. Curve filling is pretty much solved with implicit curves, stencil-and-cover or scanline-intersection-sort approaches (all of which can be done in a GPU only fashion). Stroking is where things could still improve a lot as it is almost always approximated by polygons. [0]: https://developer.nvidia.com/gpu-accelerated-path-rendering https://developer.nvidia.com/gpu-accelerated-path-rendering

raphlinus 6y ago

I think the world is going into a more layered approach, where it's the job of the driver API (Vulkan etc) to expose the power of the hardware fairly directly, and it's the job of higher levels to express the actual rendering in terms of those lower level primitives. Raytracing is an exception because you do need hardware support to do it well. Whether there's hardware that can make 2D rendering work better is an intriguing question. The NV path rendering stuff (mentioned elsethread) was an attempt (though I think it may be more driver/API than hardware), but I believe my research direction is better, in that it will be higher quality, faster, and more flexible with respect to the imaging model on standard compute shaders than an approach using the NV path rendering primitives. Obviously I have to back that up with empirical measurements, which is not yet done :)