5 ms·

Are these generated by doing normal text to (pixel) image generation, then doing a raster to svg conversion? Or are LLM models actually capable of generating SV

Are these generated by doing normal text to (pixel) image generation, then doing a raster to svg conversion? Or are LLM models actually capable of generating SVG directly? I can imagine that working for simple things: make a red circle with a blue line under it, because there are many simple examples on the web. But generating complex SVGs directly from text seems like it wouldn't work at all. The examples on their home page are well into the "too complex to generate by piecing together bits of svg examples seen during training."

bigyikes 3y ago
Definitely raster to svg conversion. The giveaway (besides complex SVG generation being too difficult, as you say) is the splotchiness of the images, since the vectorization doesn't work super well with gradients.
minimaxir 3y ago
You can prompt ChatGPT to generate SVGs. It unsurprisingly doesn't work well. I once asked it to create a SVG of Earth and it create a green ellipse on top of a blue circle.
- mewpmewp2 3y ago
  Maybe it knows earth is flat after all?
- evilduck 3y ago
  With multimodal abilities, I wonder if it could eventually check a raster of the SVG it created and then make iterative improvements that get a better result than vectorizing a raster from stable diffusion.
Zetobal 3y ago
Yes, if you look at the paths it's rather obvious.
krebby 3y ago
Yeah unfortunately differentiable SVG rendering is still something that is very difficult to accomplish with diffusion models. I looked into this last year and found VectorFusion [0][1], which appears to be using a similar technique to this app - generate the raster image and then apply a vectorization tracer. This leads to the blobbiness you see, given the way the diffusion model generates the image and the tracer applies to the output. There have been a few attempts (i.e. this recent one [2][3]) to attempt to fine-tune the parameters of the SVG with image segmentation, before comparing rendered outputs with a CLIP model. This is promising but the search space for vector images is just so huge that so far you really need to start with an existing image as a basis rather than starting from scratch. Interesting area of research! 0. https://ajayj.com/vectorfusion/ https://ajayj.com/vectorfusion/ 1. https://arxiv.org/abs/2211.11319 https://arxiv.org/abs/2211.11319 2. https://intchous.github.io/SVGCustomization/ https://intchous.github.io/SVGCustomization/ 3. https://arxiv.org/abs/2309.12302 https://arxiv.org/abs/2309.12302

bigyikes 3y ago

Definitely raster to svg conversion. The giveaway (besides complex SVG generation being too difficult, as you say) is the splotchiness of the images, since the vectorization doesn't work super well with gradients.

minimaxir 3y ago

You can prompt ChatGPT to generate SVGs. It unsurprisingly doesn't work well. I once asked it to create a SVG of Earth and it create a green ellipse on top of a blue circle.

mewpmewp2 3y ago
Maybe it knows earth is flat after all?
evilduck 3y ago
With multimodal abilities, I wonder if it could eventually check a raster of the SVG it created and then make iterative improvements that get a better result than vectorizing a raster from stable diffusion.

Zetobal 3y ago

Yes, if you look at the paths it's rather obvious.

krebby 3y ago

Yeah unfortunately differentiable SVG rendering is still something that is very difficult to accomplish with diffusion models. I looked into this last year and found VectorFusion [0][1], which appears to be using a similar technique to this app - generate the raster image and then apply a vectorization tracer. This leads to the blobbiness you see, given the way the diffusion model generates the image and the tracer applies to the output. There have been a few attempts (i.e. this recent one [2][3]) to attempt to fine-tune the parameters of the SVG with image segmentation, before comparing rendered outputs with a CLIP model. This is promising but the search space for vector images is just so huge that so far you really need to start with an existing image as a basis rather than starting from scratch. Interesting area of research! 0. https://ajayj.com/vectorfusion/ https://ajayj.com/vectorfusion/ 1. https://arxiv.org/abs/2211.11319 https://arxiv.org/abs/2211.11319 2. https://intchous.github.io/SVGCustomization/ https://intchous.github.io/SVGCustomization/ 3. https://arxiv.org/abs/2309.12302 https://arxiv.org/abs/2309.12302