12 ms·
Differentiable Rasterizer for Vector Graphics
- Jack000 6y agoThis is really cool. A possible application I see is GAN generation of new fonts.
- RicoElectrico 6y agoHowever according to the paper it can't arbitrarily change topology. > Discrete changes to the topology of vector graphics are important, but they remain beyond the scope of this paper. Still, very impressive what they achieved.
- anentropic 6y agoWhat does "topology" mean in this context? I was guessing it meant something like the z-index of different shape layers, which wouldn't pose a problem for font generation
- webmaven 6y ago> What does "topology" mean in this context? Some letterforms are not topologically equivalent and cannot be transformed into each other. For a simple example, lower case 'g' has a form in some fonts that is a loop with a descender, and in other fonts a 'two story' form where the descender connects to a lower loop.
- enriquto 6y ago> Some letterforms are not topologically equivalent and cannot be transformed into each other. But this does not seem like a fundamental limitation, just a temporary implementation detail. If you represent shapes implicitly using a condition like "u(x,y)>0", then there's no topology changes involved and you can dance between all the letters just by changing smoothly the values of u.
- gugagore 6y agoWonderful example, and this is why signed-distance functions/fields (SDF) are used. The diagram in https://en.wikipedia.org/wiki/Level-set_method https://en.wikipedia.org/wiki/Level-set_method might help explain how such a representation can "uniformly" handle topology changes. I'll excerpt: > The advantage of the level-set model is that one can perform numerical computations involving curves and surfaces on a fixed Cartesian grid without having to parameterize these objects (this is called the Eulerian approach).[1] Also, the level-set method makes it very easy to follow shapes that change topology, for example, when a shape splits in two, develops holes, or the reverse of these operations. All these make the level-set method a great tool for modeling time-varying objects, like inflation of an airbag, or a drop of oil floating in water. It is unclear to me what you mean by "implementation detail". In order for a SDF to be used in vector graphics, it must be represented symbolically (the alternative is to rasterize them, as what is implied by "level-set method", which means you must commit to a spatial resolution). But I suspect you then have the same problem on your lap: how to take derivatives with respect to structural changes in your symbolic SDF? This question reminds me a bit of https://news.ycombinator.com/item?id=19871207 https://news.ycombinator.com/item?id=19871207 discussing "Why are 2D vector graphics so much harder than 3D?"
- enriquto 6y ago> the alternative is to rasterize them, as what is implied by "level-set method", which means you must commit to a spatial resolution Rasterizing is alright, since then there are no structural changes, just independent pixel values. Notice that rasterizing the function does not commit you to an output resolution, you can always interpolate it at an arbitrarily high resolution for the rendering step.
- anentropic 6y agoah I see, thanks
- andybak 6y agoBut any shape can be made (almost) visually identical to another shape without changing the topology. For example a C can become an O if the gap becomes arbitrarily small.
- andybak 6y agoDoes anyone remember the stuff Douglas Hofstadter wrote about Metafont, human creativity and artificial intelligence? I was wondering recently whether thay might not age terribly well. Some of the things he claimed to be very difficult for a machine recently seem to be within reach.
- enriquto 6y agoAs a mathematician, I love how the word "differentiable" is changing meaning so fast for so many people! If I had been told a few years ago that the mainstream definition of "differentiable function" would become so different during my lifetime I would not have believed it! Cannot wait to teach a course in "differentiable calculus" and introduce the new stuff.
- sabertoothed 6y agoAs a non-mathematician, how did the word "differentiable" change its meaning? Is it not still that one can compute the derivative (so that Gradient Descent can be applied)? (honest question)
- sabertoothed 6y agoIf anyone else here is working on Deep Learning on/with vector graphics, feel free to ping me. I am not affiliated with the great work linked here - but I have been working on the general topic as a side project for a few years.
- andybak 6y agoI'm fairly new to the topic but I'm interested in 3D and VR and generative stuff and I've always yearned to see ways to apply crazy GAN stuff to immersive environments. One way is volumetric rendering or distance fields. I've been keeping my eye on NeRF but it seems a long way from being useful for anything realtime. Vectors and mesh representations seem somewhat neglected and this project has given me some insight into the technical problems. I've finally got CUDA running on WSL so I intend to start having a play with things. I've also got beta access to GPT-3 and I've been curious if it can do anything interesting in terms of generating structured data. Can it manipulate symbolic scene descriptions or generate meaningful vector paths? A few experiments with SVG haven't been terribly promising. I'd love to hear your thoughts on the general topic and any suggestions for things to look into.
- andybak 6y ago(you haven't mentioned how to contact you. Nothing obvious in your HN profile?)
- sabertoothed 6y agoSry, ping me here: https://twitter.com/wichmaennchen https://twitter.com/wichmaennchen