7 ms·

Blend2D – 2D Vector Graphics Engine

159 points by philix001 7y ago

royjacobs 7y ago
This is very interesting. Especially since other vector engines are either unmaintained (like Antigrain) or extremely hard to build (Skia, Cairo). I do wonder if these kind of libraries shouldn't be supplanted by hardware accelerated rendering, though.
- soylentgraham 7y ago
  This is what we really need, something that outputs an API agnostic set of render commands, (shaders, uniforms, mesh layouts etc) which auto clip themselves and such, so they can just be integrated across various engines and renderers. Of course a better format in the first place(mentioned in another comment) would just solve this anyway
  - royjacobs 7y ago
    This is similar to the approach libraries like ImGui take. They abstract the rendering backend exactly in this way.
- orbifold 7y ago
  Skia is not so terrible, you just have to use their slightly obscure build tool. And it produces >8GB build artefacts with ~50 or more dependencies... Come to think of it is a bit of a nightmare.
  - pjmlp 7y ago
    I tried to use it a few times, but having to configure a Google build environment, without Google class workstations, meant I always gave up in the process and used Skia.Sharp instead.
    - c-smile 7y ago
      Just in case: Skia can be built without Google build environment. For the Sciter (https://sciter.com https://sciter.com) I just included all needed files into projects ( MSVC and XCode ). That setup takes some initial time but after that it works as any other library like libpng, etc.
    - Matheus28 7y ago
      I explained how to build it here, give it a try: https://news.ycombinator.com/item?id=19586159 https://news.ycombinator.com/item?id=19586159
      - pjmlp 7y ago
        Thanks for the heads up, but doing it across Windows/UWP, Android and iOS requires a bit more of effort than what you described, hence Skia.Sharp instead.
        Matheus28 7y ago
        I'm doing exactly that though. To compile for iOS for example, create a separate output directory with `target_os="ios" target_cpu="arm64"`. You can see a few more examples here https://skia.org/user/build https://skia.org/user/build Unfortunately, C++ libraries are a bit harder to use than other languages, hopefully the modules proposal will make this a lot easier.
- Matheus28 7y ago
  Skia is actually not too hard to build once you're used to gn and ninja. Download both of those tools and add them to your PATH, then cd into Skia's repo (that you just cloned): $ python tools/git-sync-deps $ gn gen out/release -args="is_official_build=true is_component_build=false" $ ninja -C out/release This will build it as a static library. I believe by default it will try to use system libraries, you can make it use its own version of those libraries in this slightly more complicated process (which could be done with a script but I'm too lazy to write one right now): $ gn args out/release --list --short This will print all the current build arguments, copy all the use_system_XXX = true, you'll have to modify those to be = false, and paste them into the text editor that will open once you run: $ gn args out/release After that, run ninja to build $ ninja -C out/release For the include files, you have to add at least these to your include directory: include include/config include/core include/effects include/gpu On Windows, you might want to have two builds linking against different versions of the std lib (because of the whole iterator debug level stuff). Add these args: extra_cflags=["/MTd"] and extra_cflags=["/MT"] for debug/release builds (statically linked in this case).
- dmitriid 7y ago
  > are either unmaintained (like Antigrain) The author of Antigrain passed away a few years back, sadly. It was and still probably is one of the best graphics libraries out there, and he was really hoping for a time when antialias-free, vector-based UI on high-pixel-density screens would become reality. [1] Announcement in Russian: https://rsdn.org/forum/life/5377743.flat https://rsdn.org/forum/life/5377743.flat "Maxim Shemanarev 1966-2013. Tragically, unexpectedly passed away in his home at 47." The discussion that follows names epileptic seizure as the cause of death. [2] For example, this glorious text on text rasterization from 2007 http://www.antigrain.com/research/font_rasterization/index.html http://www.antigrain.com/research/font_rasterization/index.h...
w0utert 7y ago
Very interesting and useful, but when I read the title I was hoping it would be something different. I would love to find some kind of stripped-down vector graphics engine that is fully geared towards real-time rendering of path-based graphics for games. For example to write retro-style games that use 2D vector graphics instead of textured quads. I've been writing a simple game that uses this graphics style, and so far I've been simply using pre-rendered textures for sprites. Being able to directly render these things by composing vector-based layers would allow all kinds of interesting effects and image quality improvements, e.g. seamless scaling morphing, etc. Obviously this would be way, way slower than emulating these things using bitmap textures and/or SDF's for scaling, but for the simple graphics my game is using that should probably be fine, even on mid-range hardware there is plenty of frame time I could dedicate to vector rendering and still hit at least 30fps.
- underst0rm 7y ago
  I have had great success using the ideas presented here (http://m1el.github.io/woscope-how/index.html http://m1el.github.io/woscope-how/index.html) for simulation of traditional vector graphic screens. It is very fast and easily modified to allow for intensity modulation as well.
  - schaefer 7y ago
    I really enjoyed your demo. Thank you.
    - underst0rm 7y ago
      Just to clarify. This was not my work. I have just used the ideas presented therein for my own works and found them easy to adapt and improve upon.
- logicalshift 7y ago
  There are a few of those out there. Nanovg seems to be the one that fits your requirements most closely. If you're into Rust, lyon (plus something like gfx-rs) is another choice. It's also worth noting that the HTML canvas implementations in most browsers is also not too bad for performance. Quartz is also really fast, but Apple made it really easy to accidentally use a slow rendering path (especially for bitmap operations)
vanderZwan 7y ago
Whenever SVG rendering engines come up, sooner or later the question of "which part of the SVG spec does it render incorrectly?" comes up. How does this one fare? I get the impression that often these rendering bugs are partially caused by old optimizations that break later spec changes or missed some edge cases the first time around, so I hope that means this engine does better than most because it's more recent, built from-the-ground-up and can learn from the mistakes of others.
- mfabbri77 7y ago
  Blend2D isn't a SVG rendering engine; maybe a SVG rendering engine can be built on top of Blend2D. But anyway SVG specifications are a hell (and these include a lot of features that are not related to the pure rendering). How do i know? I'm one of the developers of https://www.amanithsvg.com https://www.amanithsvg.com
- Pxtl 7y ago
  Things like this convince me more and more that SVG is overdesigned and solving the wrong problem: The world needed a vector version of PNG far more than it needed a vector version of HTML/CSS.
  - mfabbri77 7y ago
    Definitively, this.
  - gardaani 7y ago
    W3C is creating a subset of SVG called "SVG Native". It is intended for icons and simple graphics, so it won't have support for text, scripts, links or XSL processing. Also, use of CSS or animations isn't recommended. It will be similar to SVG in OpenType fonts: https://docs.microsoft.com/en-us/typography/opentype/spec/svg https://docs.microsoft.com/en-us/typography/opentype/spec/sv...
    - pavlov 7y ago
      Wasn't there already "SVG Tiny" for this purpose? I recall it being used on mobile a long time ago, like circa 2005.
pettou 7y ago
I'm eagerly following the project for several years now, very well done Petr and Fabian!
- Asm2D 7y ago
  Really appreciated, thank you!
IshKebab 7y ago
This is such a well-made landing page. Tells me what it does, what makes it good, isn't full of vague marketing waffle, doesn't put one sentence per screen, and has actual code and output examples!
raphlinus 7y ago
How does the performance of path filling compare to font-rs?
- deleted 7y ago
  [deleted]
- Asm2D 7y ago
  Blend2D uses dense cell-buffer similarly to font-rs, however, it works quite differently and this difference allows Blend2D to be efficient even when rendering large paths: - Dense cell buffer, 32-bit integer per one cell (FreeType/Qt use sparse cell buffer and two 32-bit integers per cell, font-rs uses float if I'm not mistaken) - Blend2D builds edges before rasterization step, edge builder is optimized and performs clipping and curve flattening - Rasterization and composition happens in bands, thus storage required by the cell buffer is quite small (currently band is 32 scanlines, but we will make it adjustable based on width) - To complement dense cell buffer Blend2D uses also a shadow bit-buffer to mark cells that were altered by the rasterizer - Compositor scans bit-buffer instead of cell-buffer to quickly skip areas that were not touched by the rasterizer - Compositor is SIMD optimized and calculates multiple masks at the same time (at the moment it works at 4 pixels at a time, but this can be extended easily to 8 and 16 pixels) - Compositor clears both cell-buffer and shadow bit-buffer during composition so when the compositor finishes these buffers are zeroed and ready for another band - Blend2D maintains small zeroed memory pool that is used to quickly allocate cell and bit buffers when you create the rendering context and returned to the pool when you destroy it There are probably more differences, like parametrization of NonZero and EvenOdd fill rules, etc, but these are really implementation details to minimize the number of pipelines generated by common rendering operations. The advantage of font-rs is rendering small paths, large paths will have increasing overhead as a lot of computations would happen on zero cells. Blend2D rasterizer is universal and was tuned to perform well from small to 4K framebuffers. When I was designing Blend2D's rasterizer I wrote around 20 rasterizers and benchmarked them against each other in various resolutions. I had similar rasterizers like font-rs (but not using floats) and these were only competing in very small resolutions like 8x8 and 16x16 pixels. When shifting to larger resolutions these rasterizers would always lose as shadow bit-buffer scan is much quicker than going through zero cells, especially if you do pixel compositing. There are demo samples in bl-samples-qt repository that render animated content and have Blend2D and Qt rendering options. You can check them out to see how the rasterizer competes against Qt. Let me know if that answers your question.
c-smile 7y ago
If to speak about UI needs then CPU rasterization is just half of the problem. 320 PPI display (a.k.a. "Retina") has 9 times more pixels than old, classic 96 PPI one. So just attaching new monitor will require 10 times better CPU in order to render the same UI. That if to use CPU rasterizers only. Obviously that above is not an option. That's why Direct2D and Skia contain as GPU as CPU rasterizers. That dualism complicates things quite a lot - two alternative renderers under the same API roof shall produce similar results. So Blend2D, to be a viable solution for the UI, shall be 10 faster in rasterizing than any current alternatives. Yet, it was NV_path_rendering OpenGL extension from NVIDIA aimed for 2D path rasterization, but it seems the effort is dimmed now as OpenGL itself. OpenGL architecture, that was created to run H/W accelerated full screen apps, is far from being adequate for windowed UI. So far Microsoft's Direct2D is the best thing that we have for H/W accelerated UI so far. And WARP mode in Direct2D (CPU rasterizer) is pretty close to the Blend2D - they also use JIT for rasterizing AFAIK.
- Asm2D 7y ago
  It's true that increasing the size of framebuffer demands more from CPU as well. According to my experience a single core on a modern machine has no problem to render real-time into a FullHD framebuffer at high frame rate (depending on the content of course, but UI is fine). This means that multithreaded renderers using 4 threads should be able to render to 4K framebuffer without any issues. Since AMD will release 16c/32t consumer CPUs this year I see no problem on this front as we will have the computational power to run several multithreaded renderers at the same time. Blend2D has multithreaded rendering on roadmap - I have experience in this topic and everything in Blend2D was designed with multithreading in mind (banding for example). The implementation I'm planning would scale very well. NV_path_rendering - I haven't seen any detailed comparison to be honest. Frame-rate is not enough to compare CPU vs GPU technology - both memory consumption and power consumption are important as well to calculate frame-rate per watt. I cannot comment on Direct2D as it's not open source and it runs only on a single operating system. So I don't consider Direct2D as a competition at the moment.
  - c-smile 7y ago
    Well, CPU rasterization is always O(N) complex (where N is number of pixels on screen). Multithreading here just adds constant multiplier that according to the math will still lead to O(N). While GPU rasterization, from application perspective, is near O(1) - does not depend on number of pixels in ideal circumstances. And having multiple threads to render UI is not desired - there are too many CPU consumers on modern desktop, e.g. online radio that is playing now, etc. I am not saying that CPU rasterization makes no sense. Quite contrary. As a practical example: in Sciter on Linux/GTK I am using Cairo backend by default as OpenGL inside GTK windows is horrible. So Skia does not help there at all - Cairo and its CPU rasterizer is used. If we would have something that allows to rasterize paths 5-10 times faster than current Cairo - it will solve all current desktop needs I think. In principle 192 PPI resolution for desktop monitors of practical sizes (24 inch, 3840x2160 pixels) is OK - human eye will not be able to see separate pixels. Pretty much the same number of pixels is on mobiles ( iPadPro: 2732x2048 ). These are targets that need to be considered. Practical requirements: Take HN site in browser. Open it full screen. Decent 2D library should be able to rasterize that amount of text with 60 FPS (e.g. kinetic scrolling).

159 points by philix001 7y ago

This is very interesting. Especially since other vector engines are either unmaintained (like Antigrain) or extremely hard to build (Skia, Cairo). I do wonder if these kind of libraries shouldn't be supplanted by hardware accelerated rendering, though.

soylentgraham 7y ago
This is what we really need, something that outputs an API agnostic set of render commands, (shaders, uniforms, mesh layouts etc) which auto clip themselves and such, so they can just be integrated across various engines and renderers. Of course a better format in the first place(mentioned in another comment) would just solve this anyway
- royjacobs 7y ago
  This is similar to the approach libraries like ImGui take. They abstract the rendering backend exactly in this way.
orbifold 7y ago
Skia is not so terrible, you just have to use their slightly obscure build tool. And it produces >8GB build artefacts with ~50 or more dependencies... Come to think of it is a bit of a nightmare.
- pjmlp 7y ago
  I tried to use it a few times, but having to configure a Google build environment, without Google class workstations, meant I always gave up in the process and used Skia.Sharp instead.
  - c-smile 7y ago
    Just in case: Skia can be built without Google build environment. For the Sciter (https://sciter.com https://sciter.com) I just included all needed files into projects ( MSVC and XCode ). That setup takes some initial time but after that it works as any other library like libpng, etc.
  - Matheus28 7y ago
    I explained how to build it here, give it a try: https://news.ycombinator.com/item?id=19586159 https://news.ycombinator.com/item?id=19586159
    - pjmlp 7y ago
      Thanks for the heads up, but doing it across Windows/UWP, Android and iOS requires a bit more of effort than what you described, hence Skia.Sharp instead.
      - Matheus28 7y ago
        I'm doing exactly that though. To compile for iOS for example, create a separate output directory with `target_os="ios" target_cpu="arm64"`. You can see a few more examples here https://skia.org/user/build https://skia.org/user/build Unfortunately, C++ libraries are a bit harder to use than other languages, hopefully the modules proposal will make this a lot easier.
Matheus28 7y ago
Skia is actually not too hard to build once you're used to gn and ninja. Download both of those tools and add them to your PATH, then cd into Skia's repo (that you just cloned): $ python tools/git-sync-deps $ gn gen out/release -args="is_official_build=true is_component_build=false" $ ninja -C out/release This will build it as a static library. I believe by default it will try to use system libraries, you can make it use its own version of those libraries in this slightly more complicated process (which could be done with a script but I'm too lazy to write one right now): $ gn args out/release --list --short This will print all the current build arguments, copy all the use_system_XXX = true, you'll have to modify those to be = false, and paste them into the text editor that will open once you run: $ gn args out/release After that, run ninja to build $ ninja -C out/release For the include files, you have to add at least these to your include directory: include include/config include/core include/effects include/gpu On Windows, you might want to have two builds linking against different versions of the std lib (because of the whole iterator debug level stuff). Add these args: extra_cflags=["/MTd"] and extra_cflags=["/MT"] for debug/release builds (statically linked in this case).
dmitriid 7y ago
> are either unmaintained (like Antigrain) The author of Antigrain passed away a few years back, sadly. It was and still probably is one of the best graphics libraries out there, and he was really hoping for a time when antialias-free, vector-based UI on high-pixel-density screens would become reality. [1] Announcement in Russian: https://rsdn.org/forum/life/5377743.flat https://rsdn.org/forum/life/5377743.flat "Maxim Shemanarev 1966-2013. Tragically, unexpectedly passed away in his home at 47." The discussion that follows names epileptic seizure as the cause of death. [2] For example, this glorious text on text rasterization from 2007 http://www.antigrain.com/research/font_rasterization/index.html http://www.antigrain.com/research/font_rasterization/index.h...

w0utert 7y ago

Very interesting and useful, but when I read the title I was hoping it would be something different. I would love to find some kind of stripped-down vector graphics engine that is fully geared towards real-time rendering of path-based graphics for games. For example to write retro-style games that use 2D vector graphics instead of textured quads. I've been writing a simple game that uses this graphics style, and so far I've been simply using pre-rendered textures for sprites. Being able to directly render these things by composing vector-based layers would allow all kinds of interesting effects and image quality improvements, e.g. seamless scaling morphing, etc. Obviously this would be way, way slower than emulating these things using bitmap textures and/or SDF's for scaling, but for the simple graphics my game is using that should probably be fine, even on mid-range hardware there is plenty of frame time I could dedicate to vector rendering and still hit at least 30fps.

underst0rm 7y ago
I have had great success using the ideas presented here (http://m1el.github.io/woscope-how/index.html http://m1el.github.io/woscope-how/index.html) for simulation of traditional vector graphic screens. It is very fast and easily modified to allow for intensity modulation as well.
- schaefer 7y ago
  I really enjoyed your demo. Thank you.
  - underst0rm 7y ago
    Just to clarify. This was not my work. I have just used the ideas presented therein for my own works and found them easy to adapt and improve upon.
logicalshift 7y ago
There are a few of those out there. Nanovg seems to be the one that fits your requirements most closely. If you're into Rust, lyon (plus something like gfx-rs) is another choice. It's also worth noting that the HTML canvas implementations in most browsers is also not too bad for performance. Quartz is also really fast, but Apple made it really easy to accidentally use a slow rendering path (especially for bitmap operations)

vanderZwan 7y ago

Whenever SVG rendering engines come up, sooner or later the question of "which part of the SVG spec does it render incorrectly?" comes up. How does this one fare? I get the impression that often these rendering bugs are partially caused by old optimizations that break later spec changes or missed some edge cases the first time around, so I hope that means this engine does better than most because it's more recent, built from-the-ground-up and can learn from the mistakes of others.

mfabbri77 7y ago
Blend2D isn't a SVG rendering engine; maybe a SVG rendering engine can be built on top of Blend2D. But anyway SVG specifications are a hell (and these include a lot of features that are not related to the pure rendering). How do i know? I'm one of the developers of https://www.amanithsvg.com https://www.amanithsvg.com
Pxtl 7y ago
Things like this convince me more and more that SVG is overdesigned and solving the wrong problem: The world needed a vector version of PNG far more than it needed a vector version of HTML/CSS.
- mfabbri77 7y ago
  Definitively, this.
- gardaani 7y ago
  W3C is creating a subset of SVG called "SVG Native". It is intended for icons and simple graphics, so it won't have support for text, scripts, links or XSL processing. Also, use of CSS or animations isn't recommended. It will be similar to SVG in OpenType fonts: https://docs.microsoft.com/en-us/typography/opentype/spec/svg https://docs.microsoft.com/en-us/typography/opentype/spec/sv...
  - pavlov 7y ago
    Wasn't there already "SVG Tiny" for this purpose? I recall it being used on mobile a long time ago, like circa 2005.

pettou 7y ago

I'm eagerly following the project for several years now, very well done Petr and Fabian!

Asm2D 7y ago
Really appreciated, thank you!

IshKebab 7y ago

This is such a well-made landing page. Tells me what it does, what makes it good, isn't full of vague marketing waffle, doesn't put one sentence per screen, and has actual code and output examples!

raphlinus 7y ago

How does the performance of path filling compare to font-rs?

deleted 7y ago
[deleted]
Asm2D 7y ago
Blend2D uses dense cell-buffer similarly to font-rs, however, it works quite differently and this difference allows Blend2D to be efficient even when rendering large paths: - Dense cell buffer, 32-bit integer per one cell (FreeType/Qt use sparse cell buffer and two 32-bit integers per cell, font-rs uses float if I'm not mistaken) - Blend2D builds edges before rasterization step, edge builder is optimized and performs clipping and curve flattening - Rasterization and composition happens in bands, thus storage required by the cell buffer is quite small (currently band is 32 scanlines, but we will make it adjustable based on width) - To complement dense cell buffer Blend2D uses also a shadow bit-buffer to mark cells that were altered by the rasterizer - Compositor scans bit-buffer instead of cell-buffer to quickly skip areas that were not touched by the rasterizer - Compositor is SIMD optimized and calculates multiple masks at the same time (at the moment it works at 4 pixels at a time, but this can be extended easily to 8 and 16 pixels) - Compositor clears both cell-buffer and shadow bit-buffer during composition so when the compositor finishes these buffers are zeroed and ready for another band - Blend2D maintains small zeroed memory pool that is used to quickly allocate cell and bit buffers when you create the rendering context and returned to the pool when you destroy it There are probably more differences, like parametrization of NonZero and EvenOdd fill rules, etc, but these are really implementation details to minimize the number of pipelines generated by common rendering operations. The advantage of font-rs is rendering small paths, large paths will have increasing overhead as a lot of computations would happen on zero cells. Blend2D rasterizer is universal and was tuned to perform well from small to 4K framebuffers. When I was designing Blend2D's rasterizer I wrote around 20 rasterizers and benchmarked them against each other in various resolutions. I had similar rasterizers like font-rs (but not using floats) and these were only competing in very small resolutions like 8x8 and 16x16 pixels. When shifting to larger resolutions these rasterizers would always lose as shadow bit-buffer scan is much quicker than going through zero cells, especially if you do pixel compositing. There are demo samples in bl-samples-qt repository that render animated content and have Blend2D and Qt rendering options. You can check them out to see how the rasterizer competes against Qt. Let me know if that answers your question.

c-smile 7y ago

If to speak about UI needs then CPU rasterization is just half of the problem. 320 PPI display (a.k.a. "Retina") has 9 times more pixels than old, classic 96 PPI one. So just attaching new monitor will require 10 times better CPU in order to render the same UI. That if to use CPU rasterizers only. Obviously that above is not an option. That's why Direct2D and Skia contain as GPU as CPU rasterizers. That dualism complicates things quite a lot - two alternative renderers under the same API roof shall produce similar results. So Blend2D, to be a viable solution for the UI, shall be 10 faster in rasterizing than any current alternatives. Yet, it was NV_path_rendering OpenGL extension from NVIDIA aimed for 2D path rasterization, but it seems the effort is dimmed now as OpenGL itself. OpenGL architecture, that was created to run H/W accelerated full screen apps, is far from being adequate for windowed UI. So far Microsoft's Direct2D is the best thing that we have for H/W accelerated UI so far. And WARP mode in Direct2D (CPU rasterizer) is pretty close to the Blend2D - they also use JIT for rasterizing AFAIK.

Asm2D 7y ago
It's true that increasing the size of framebuffer demands more from CPU as well. According to my experience a single core on a modern machine has no problem to render real-time into a FullHD framebuffer at high frame rate (depending on the content of course, but UI is fine). This means that multithreaded renderers using 4 threads should be able to render to 4K framebuffer without any issues. Since AMD will release 16c/32t consumer CPUs this year I see no problem on this front as we will have the computational power to run several multithreaded renderers at the same time. Blend2D has multithreaded rendering on roadmap - I have experience in this topic and everything in Blend2D was designed with multithreading in mind (banding for example). The implementation I'm planning would scale very well. NV_path_rendering - I haven't seen any detailed comparison to be honest. Frame-rate is not enough to compare CPU vs GPU technology - both memory consumption and power consumption are important as well to calculate frame-rate per watt. I cannot comment on Direct2D as it's not open source and it runs only on a single operating system. So I don't consider Direct2D as a competition at the moment.
- c-smile 7y ago
  Well, CPU rasterization is always O(N) complex (where N is number of pixels on screen). Multithreading here just adds constant multiplier that according to the math will still lead to O(N). While GPU rasterization, from application perspective, is near O(1) - does not depend on number of pixels in ideal circumstances. And having multiple threads to render UI is not desired - there are too many CPU consumers on modern desktop, e.g. online radio that is playing now, etc. I am not saying that CPU rasterization makes no sense. Quite contrary. As a practical example: in Sciter on Linux/GTK I am using Cairo backend by default as OpenGL inside GTK windows is horrible. So Skia does not help there at all - Cairo and its CPU rasterizer is used. If we would have something that allows to rasterize paths 5-10 times faster than current Cairo - it will solve all current desktop needs I think. In principle 192 PPI resolution for desktop monitors of practical sizes (24 inch, 3840x2160 pixels) is OK - human eye will not be able to see separate pixels. Pretty much the same number of pixels is on mobiles ( iPadPro: 2732x2048 ). These are targets that need to be considered. Practical requirements: Take HN site in browser. Open it full screen. Decent 2D library should be able to rasterize that amount of text with 60 FPS (e.g. kinetic scrolling).