4 ms·

A novel technological approach to vector graphics (“Vector Graphics Complexes”)

23 points by simonrepp 5y ago

ninetenfour 5y ago
Amazing, but this really should be a web-based app and not desktop based. Or a hybrid strategy where the desktop-based app is actually running JavaScript in an Electron shell or something. Please switch your approach ASAP or you will be beaten by a web-based app that does this exact thing, someone else will be the Figma to your Sketch. This specific use-case doesn't require the large amount of resources that would necessitate a desktop app.
- borisdalstein 5y ago
  Thanks for the comment, I'm the developer of the app! Having it run in a browser would clearly be convenient, but unfortunately Javascript is really too slow. I even wish C++ was faster. Rendering the vector paths requires the same type of heavy computation as a modern AAA game, and we don't see AAA games implemented in Javascript for good reasons. The paths need to be tessellated, sent to the GPU, then the different layers need to be composited, and all of this takes computational resources. In addition, unlike Figma where the vector paths are created by manipulating control points, in VGC Illustration it is expected that the paths can be created by sketching them, hand-drawn. This requires extremely low latency and high FPS for the drawing experience to be any good. There's already quite a bit of lag between the stylus and the shown stroke even in Photoshop, you don't wan't this to be any worse :) I hope this makes sense.
  - Const-me 5y ago
    How are you rendering these vectors on Linux? I'm asking because AFAIK unlike Windows with Direct2D, Linux doesn't have GPU-targeted vector graphics libraries. I have once made my own one when I needed something similar: https://github.com/Const-me/Vrmac#vector-graphics-engine https://github.com/Const-me/Vrmac#vector-graphics-engine
    - borisdalstein 5y ago
      I'm using 3D graphics APIs, OpenGL on Linux currently, but Vulkan will be an option later. I'm converting everything to triangles on the CPU and send to OpenGL. Note that the animation app actually needs to show 3D graphics for a 2D+time visualization which is part of the innovation in my research.
      - Const-me 5y ago
        > I'm converting everything to triangles on the CPU and send to OpenGL Thanks, that’s what I thought. The library I have linked does the same thing. Only it’s either D3D12, or GLES 3.1 underneath.
        borisdalstein 5y ago
        Wow, your library actually looks impressive! Yeah, I'm pretty much implementing all the same things.
  - ninetenfour 5y ago
    I think you can tessellation on the CPU in an adaptive fashion (e.g. based on curvature or similar) and update that on a per frame (add, remove tessellation points) basis rather than re-tesselating from scratch each frame. The tessellation would just have weights from each control point. You can modify the vertices on the CPU, update the tessellation points and update those to the GPU. Get 60fps should be possible even when updating 1Ks of vertices (probably more) per second. Do this for whatever changed using dirty lists or something so you do not update everything to the GPU on each frame. This should be relatively fast if you using ArrayBuffers. 100% this can be done in JavaScript in a very fast fashion. It may only seem like it can not be done if you are doing JS poorly. Maybe people make the mistake of not using ArrayBuffers for everything and that makes it slow. The main limitation in JS is the lack of really great multithreading, but I do not think you need it here. If you want ultimate speed use WASM for the CPU-based updating of your geometry structures. Anyhow, you are mistaken in your current beliefs and it will hurt you going forward.
    - Const-me 5y ago
      > I think you can tessellation on the CPU in an adaptive fashion (e.g. based on curvature or similar) and update that on a per frame (add, remove tessellation points) basis rather than re-tesselating from scratch each frame. The GPU’s hardware-implemented tessellation is (a) not compatible enough. It’s OK on Windows because Microsoft requires them for GPU vendors to declare the support of Direct3D 11. On the rest of the platforms, support across GPU vendors varies. And (b) doesn’t help much for 2D vector graphics. The hardware tessellation can be good for terrain, trees, or other triangle meshes in 3D space. Doesn’t help much for these Bezier curves/elliptical arcs for 2D shapes. Especially so for stroked paths. Counter-intuitively, stroked paths are harder to render than filled ones. The offset of Bezier spline is not representable as another Bezier spline. Also, strokes have more parameters on input: line caps, join types, dashes, miter limit, etc. > The main limitation in JS is the lack of really great multithreading Also lack of SIMD. Also, the code in general is slow compared to C++, C# and many other statically typed languages. It’s incredibly hard to generate fast code from very dynamic languages like JS or Python, where everything is a hash map. > I do not think you need it here. Here’s my code which offloads CPU-bound pieces of 2D rendering to other CPU cores: https://github.com/Const-me/Vrmac/tree/master/Vrmac/Draw/Tessellate https://github.com/Const-me/Vrmac/tree/master/Vrmac/Draw/Tes... Multithreading helped a lot.
  - jlokier 5y ago
    Just in case it's useful: You actually can run C++ in the browser these days to do the intensive things, by using WebAssembly (WASM). It can call out to graphics APIs, so it might work for this project without changing the C++ much.
- wds 5y ago
  But why should it be web-based? Why would something else web-based 'beat' this based on that fact alone?
  - borisdalstein 5y ago
    Playing devil's advocate, I'd argue that it does make development much faster, and portability much easier. This is important. But otherwise I clearly agree with you. There's no reason why I would make it web-based in the first place.
  - webmaven 5y ago
    > But why should it be web-based? Why would something else web-based 'beat' this based on that fact alone? Software distribution and/or installation, presumably. But these factors mostly apply to the consumer and prosumer market segments (hence, the web triggered the 'consumerization of IT'), rather than to the professional market.
adamrezich 5y ago
this looks very cool, I've long wondered why vector graphics tools seem to have largely stagnated in terms of features and such.
- borisdalstein 5y ago
  It is indeed strange, the technology hasn't progressed much, especially compared to 3D modeling/animation! I hypothesize that the stagnation is because researchers in computer science are more naturally drawn to 3D since the problems seem more fun and challenging, and they may not realize there are still things to improve in the 2D world.
fractallyte 5y ago
Back in the early 2000s, there was a vector graphics application called Expression, created by Creature House, a small Hong Kong-based company (https://en.wikipedia.org/wiki/Creature_House_Expression https://en.wikipedia.org/wiki/Creature_House_Expression). They also had a beta of an animation version of Expression, called LivingCels. It had been battle-tested in various local animation studios, and was just getting ready to burst into the market... (dominated, at that time, by Adobe Illustrator and Flash) Expression was based on the founders' PhD work too: 'skeletal strokes', a really cool feature that allowed painting with complex vector or bitmap strokes. It reached version 3, then Microsoft acquired the company in 2003, re-wrote the software, got it wrong... and quietly killed it. Expression had the BEST user interface, was packed with features, and the website was whimsical and unique. Guess what? It's still available! A free download for Windows, that works perfectly in Wine. It's better than Affinity Designer, and most other vector drawing apps. I would place it alongside Adobe Illustrator and CorelDRAW - except it's more intuitive and easier to use. (Sure, it lacks features, because it's nearly 20 years gone!) I love this new vector graphics technology. But they have a lot of catching up to do to make a full-featured application. If only their tech could be combined with Expression (a far-off fantasy!), the world would have another awesome vector graphics application.
- borisdalstein 5y ago
  Wow, this is cool, I had heard of Expression but didn't the whole backstory. I will definitely have to check it out. And indeed I have a lot of catching up to do feature-wise :) VGC Illustration won't be able to be as feature-complete as Illustrator for a long time, but hopefully the killer features and better drawing experience will make up for it.
  - fractallyte 5y ago
    I sincerely wish you well! (And I'll back your KS campaign) I've been entertaining a thought for a long time... to approach MS and ask (very, very nicely!) if they'd be willing to part with the source code for Expression and LivingCels. They're not using it in any way, and these are killer apps which definitely have a lot of life in them yet.

23 points by simonrepp 5y ago

Amazing, but this really should be a web-based app and not desktop based. Or a hybrid strategy where the desktop-based app is actually running JavaScript in an Electron shell or something. Please switch your approach ASAP or you will be beaten by a web-based app that does this exact thing, someone else will be the Figma to your Sketch. This specific use-case doesn't require the large amount of resources that would necessitate a desktop app.

borisdalstein 5y ago
Thanks for the comment, I'm the developer of the app! Having it run in a browser would clearly be convenient, but unfortunately Javascript is really too slow. I even wish C++ was faster. Rendering the vector paths requires the same type of heavy computation as a modern AAA game, and we don't see AAA games implemented in Javascript for good reasons. The paths need to be tessellated, sent to the GPU, then the different layers need to be composited, and all of this takes computational resources. In addition, unlike Figma where the vector paths are created by manipulating control points, in VGC Illustration it is expected that the paths can be created by sketching them, hand-drawn. This requires extremely low latency and high FPS for the drawing experience to be any good. There's already quite a bit of lag between the stylus and the shown stroke even in Photoshop, you don't wan't this to be any worse :) I hope this makes sense.
- Const-me 5y ago
  How are you rendering these vectors on Linux? I'm asking because AFAIK unlike Windows with Direct2D, Linux doesn't have GPU-targeted vector graphics libraries. I have once made my own one when I needed something similar: https://github.com/Const-me/Vrmac#vector-graphics-engine https://github.com/Const-me/Vrmac#vector-graphics-engine
  - borisdalstein 5y ago
    I'm using 3D graphics APIs, OpenGL on Linux currently, but Vulkan will be an option later. I'm converting everything to triangles on the CPU and send to OpenGL. Note that the animation app actually needs to show 3D graphics for a 2D+time visualization which is part of the innovation in my research.
    - Const-me 5y ago
      > I'm converting everything to triangles on the CPU and send to OpenGL Thanks, that’s what I thought. The library I have linked does the same thing. Only it’s either D3D12, or GLES 3.1 underneath.
      - borisdalstein 5y ago
        Wow, your library actually looks impressive! Yeah, I'm pretty much implementing all the same things.
- ninetenfour 5y ago
  I think you can tessellation on the CPU in an adaptive fashion (e.g. based on curvature or similar) and update that on a per frame (add, remove tessellation points) basis rather than re-tesselating from scratch each frame. The tessellation would just have weights from each control point. You can modify the vertices on the CPU, update the tessellation points and update those to the GPU. Get 60fps should be possible even when updating 1Ks of vertices (probably more) per second. Do this for whatever changed using dirty lists or something so you do not update everything to the GPU on each frame. This should be relatively fast if you using ArrayBuffers. 100% this can be done in JavaScript in a very fast fashion. It may only seem like it can not be done if you are doing JS poorly. Maybe people make the mistake of not using ArrayBuffers for everything and that makes it slow. The main limitation in JS is the lack of really great multithreading, but I do not think you need it here. If you want ultimate speed use WASM for the CPU-based updating of your geometry structures. Anyhow, you are mistaken in your current beliefs and it will hurt you going forward.
  - Const-me 5y ago
    > I think you can tessellation on the CPU in an adaptive fashion (e.g. based on curvature or similar) and update that on a per frame (add, remove tessellation points) basis rather than re-tesselating from scratch each frame. The GPU’s hardware-implemented tessellation is (a) not compatible enough. It’s OK on Windows because Microsoft requires them for GPU vendors to declare the support of Direct3D 11. On the rest of the platforms, support across GPU vendors varies. And (b) doesn’t help much for 2D vector graphics. The hardware tessellation can be good for terrain, trees, or other triangle meshes in 3D space. Doesn’t help much for these Bezier curves/elliptical arcs for 2D shapes. Especially so for stroked paths. Counter-intuitively, stroked paths are harder to render than filled ones. The offset of Bezier spline is not representable as another Bezier spline. Also, strokes have more parameters on input: line caps, join types, dashes, miter limit, etc. > The main limitation in JS is the lack of really great multithreading Also lack of SIMD. Also, the code in general is slow compared to C++, C# and many other statically typed languages. It’s incredibly hard to generate fast code from very dynamic languages like JS or Python, where everything is a hash map. > I do not think you need it here. Here’s my code which offloads CPU-bound pieces of 2D rendering to other CPU cores: https://github.com/Const-me/Vrmac/tree/master/Vrmac/Draw/Tessellate https://github.com/Const-me/Vrmac/tree/master/Vrmac/Draw/Tes... Multithreading helped a lot.
- jlokier 5y ago
  Just in case it's useful: You actually can run C++ in the browser these days to do the intensive things, by using WebAssembly (WASM). It can call out to graphics APIs, so it might work for this project without changing the C++ much.
wds 5y ago
But why should it be web-based? Why would something else web-based 'beat' this based on that fact alone?
- borisdalstein 5y ago
  Playing devil's advocate, I'd argue that it does make development much faster, and portability much easier. This is important. But otherwise I clearly agree with you. There's no reason why I would make it web-based in the first place.
- webmaven 5y ago
  > But why should it be web-based? Why would something else web-based 'beat' this based on that fact alone? Software distribution and/or installation, presumably. But these factors mostly apply to the consumer and prosumer market segments (hence, the web triggered the 'consumerization of IT'), rather than to the professional market.

adamrezich 5y ago

this looks very cool, I've long wondered why vector graphics tools seem to have largely stagnated in terms of features and such.

borisdalstein 5y ago
It is indeed strange, the technology hasn't progressed much, especially compared to 3D modeling/animation! I hypothesize that the stagnation is because researchers in computer science are more naturally drawn to 3D since the problems seem more fun and challenging, and they may not realize there are still things to improve in the 2D world.

fractallyte 5y ago

Back in the early 2000s, there was a vector graphics application called Expression, created by Creature House, a small Hong Kong-based company (https://en.wikipedia.org/wiki/Creature_House_Expression https://en.wikipedia.org/wiki/Creature_House_Expression). They also had a beta of an animation version of Expression, called LivingCels. It had been battle-tested in various local animation studios, and was just getting ready to burst into the market... (dominated, at that time, by Adobe Illustrator and Flash) Expression was based on the founders' PhD work too: 'skeletal strokes', a really cool feature that allowed painting with complex vector or bitmap strokes. It reached version 3, then Microsoft acquired the company in 2003, re-wrote the software, got it wrong... and quietly killed it. Expression had the BEST user interface, was packed with features, and the website was whimsical and unique. Guess what? It's still available! A free download for Windows, that works perfectly in Wine. It's better than Affinity Designer, and most other vector drawing apps. I would place it alongside Adobe Illustrator and CorelDRAW - except it's more intuitive and easier to use. (Sure, it lacks features, because it's nearly 20 years gone!) I love this new vector graphics technology. But they have a lot of catching up to do to make a full-featured application. If only their tech could be combined with Expression (a far-off fantasy!), the world would have another awesome vector graphics application.

borisdalstein 5y ago
Wow, this is cool, I had heard of Expression but didn't the whole backstory. I will definitely have to check it out. And indeed I have a lot of catching up to do feature-wise :) VGC Illustration won't be able to be as feature-complete as Illustrator for a long time, but hopefully the killer features and better drawing experience will make up for it.
- fractallyte 5y ago
  I sincerely wish you well! (And I'll back your KS campaign) I've been entertaining a thought for a long time... to approach MS and ask (very, very nicely!) if they'd be willing to part with the source code for Expression and LivingCels. They're not using it in any way, and these are killer apps which definitely have a lot of life in them yet.