3 ms·
Sure, they're the wrong tool for drawing a pelican - but testing their SVG output is a useful way to get a feel for how good they are at step by step reasoning,
by simonw 2y ago
Sure, they're the wrong tool for drawing a pelican - but testing their SVG output is a useful way to get a feel for how good they are at step by step reasoning, coordinate systems, spatial awareness and generating valid SVG/XML.
There are genuinely useful applications of SVG-generation from LLMs - outputting simple infographics or charts for example.
I use LLMs to write HTML all the time, of which SVG is a useful optional component.
- KMnO4 2y agoThis benchmark is interesting, because it sidesteps the reasoning and process that humans would excel at. For example, if I asked you to assemble a bookshelf with some wood, nails, and cement, you might first make a hammer with the cement before trying to assemble the bookshelf. You can get a much better image by first asking the (multimodal) LLM to draw an image of a pelican on a bicycle, and then generate an SVG using the referenced image. https://chatgpt.com/share/67609300-9abc-800d-9b26-95074f214954 https://chatgpt.com/share/67609300-9abc-800d-9b26-95074f2149...