3 ms·
That's possibly the worst pelican I saw from all recent LLMs. Meanwhile GLM 5.2 drew a cool self-contained fully animated SVG pelican: https://simonwillison.n
by bel8 4mo ago
That's possibly the worst pelican I saw from all recent LLMs.
Meanwhile GLM 5.2 drew a cool self-contained fully animated SVG pelican:
https://simonwillison.net/2026/Jun/17/glm-52 https://simonwillison.net/2026/Jun/17/glm-52
- simonw 4mo agoYeah, GLM have been beating Anthropic on the pelicans for a while now. (I suspect that's more of an indication that Anthropic have chosen not to waste resources training on animals riding vehicles, personally.)
- bel8 4mo agoThat's one possibility albeit quite charitable. I'd be inclined to think the same personally if GLM 5.2 wasn't also rocking in other areas too.
- kamranjon 4mo agoThis is interesting, I haven’t actually heard you suggest that the labs are focusing on this benchmark before. Have you come around to this position as a result of the quality of pelicans you’ve been getting? The reason I thought this was an interesting benchmark is because it’s a non-image generating model creating an image using SVG code, so it kinda spans capabilities. If an AI lab trained a model specifically for animals riding bicycles it seem trivial to modify the prompt and determine if it was trained specifically for that or if it’s generalized a skill and can also generate a proper orangutan walking on stilts or an armadillo on a skateboard, this sort of thing?
- simonw 4mo agoThat's what I've been doing - trying different animals in different vehicles. I'd love to find a lab who does a good pelican on a bicycle but sucks at other combinations, but sadly that's not happened yet. Google Gemini have openly boasted about their animals on vehicles results! https://x.com/JeffDean/status/2024525132266688757 https://x.com/JeffDean/status/2024525132266688757
- philipwhiuk 4mo agoJust need the legs to interact with the pedals now :D
- bel8 4mo agoThat's something that Sonnet 5 couldn't get right even on a static SVG.