5 ms·
Accessed via OpenRouter, this one decided to wrap the SVG pelican in HTML with controls for the animation speed: https://gisthost.github.io/?ecaad98efe0f747e27b
by simonw 7mo ago
Accessed via OpenRouter, this one decided to wrap the SVG pelican in HTML with controls for the animation speed: https://gisthost.github.io/?ecaad98efe0f747e27bc0e0ebc669e94/pelican.html https://gisthost.github.io/?ecaad98efe0f747e27bc0e0ebc669e94...
Transcript and HTML here: https://gist.github.com/simonw/ecaad98efe0f747e27bc0e0ebc669e94#2026-04-20t164936----conversation-01kpnwt8d2bt5qwkm60j9sbkbs-id-01kpnwra0prz6v822cct5b08kq https://gist.github.com/simonw/ecaad98efe0f747e27bc0e0ebc669...
- SwellJoe 7mo agoWe got an overachiever, here. Kimi sounds like a teacher's pet kind of name.
- subscribed 7mo agoUnderappreciated comment
- FlyingSnake 7mo agoAt this point drawing these Pelicans must be in the training data sets.
- ffsm8 7mo agoClearly not. I mean the prompt was succinct and clear, as always - and it still decided to hallucinate multiple features (animation + controls) beyond the prompt. It'd also like to point out that to date no drawing was actually good from an actual quality perspective (as in comparative to what a decent designer would throw together) Theyre always only "good" from the perspective of it being a one shot low effort prompt. Very little content for training purposes.
- nwienert 7mo agoThe way I’ve come to think of LLM is that what the produce in a single reply even with thinking turned up, is akin to what you’d do in a single short session of work. And so if you ask it to do something big it will do a very surface level implementation. But if you have it iterate many times, or give it small pieces each time, you’ll end up with something closer to what a human would do. I imagine the pelican test but done in a harness that has the agents iterate 10+ times would be closer to what you’d expect, especially if a visual model was critiquing each time.
- slopinthebag 7mo agoYeah, this is how I use AI. Instead of a single session one-shot, it's usually limited to single targeted edits, and then I steer it on each step. Takes longer but the output is actually what I want.
- serial_dev 7mo agoWhat does good even mean… I have no idea what a good “pelican on a bike” should look like. It’s a fun prompt because there is no good answers… at least so I thought.
- abustamam 7mo agoYeah that was exactly Simon's intent. https://simonwillison.net/2025/Nov/13/training-for-pelicans-riding-bicycles/ https://simonwillison.net/2025/Nov/13/training-for-pelicans-...
- ffsm8 7mo agoThere are countless examples of animals riding bicycles etc from Comic books I grew up with It would always look goofy - by design, but it usually looked good.
- scosman 7mo agonot if I can help it! https://github.com/scosman/pelicans_riding_bicycles https://github.com/scosman/pelicans_riding_bicycles
- icelancer 7mo agolove this adversarial work
- smcleod 7mo agoThis is pretty funny
- AmbroseBierce 7mo agoI hereby certify that these are indeed the most perfect and precise svg depictions of pelican riding a bicycle, also known among biology scholars as pelycles
- ahmadyan 7mo agoI love it!
- justinclift 7mo agoThat's truly a wonderful collection of pelicans riding bicycles. Much Win! ;)
- wvlia5 7mo agoJust a few years ago, this would have been a meaningless repo.
- 7mo ago
- BrokenCogs 7mo agoYes we all know that, but we still like to see the pelicans because it's a tradition more or less
- alfiedotwtf 7mo agoWhy no Utah Teapot!
- abustamam 7mo agoCould be! Simon wrote about that here though https://simonwillison.net/2025/Nov/13/training-for-pelicans-riding-bicycles/ https://simonwillison.net/2025/Nov/13/training-for-pelicans-...
- stingraycharles 7mo ago> If a model finally comes out that produces an excellent SVG of a pelican riding a bicycle you can bet I’m going to test it on all manner of creatures riding all sorts of transportation devices. This relies on the false premise that, if they would include it in their training dataset, it would be perfect. All they need to do is be good enough and better than the other, not perfect.
- abustamam 7mo agoI'm not sure if we can have a "perfect" Pelican riding a bicycle. Like, I could probably commission a highly experienced artist to draw one and I don't think it would be perfect. The legs would probably have to be too long, or pedals oddly placed, or handles strange, or wings with hands. Based on the one Simon commented though, I'd say we're in decent territory to try the latter part of his hypothesis.
- ethbr1 7mo ago> The legs would probably have to be too long, or pedals oddly placed, or handles strange, or wings with hands. In all seriousness, that's what makes it an interesting test: it's asking for something technically impossible, that requires artistic license to make coherent. Making specific choices on where to bend reality (and where not to) is a big chunk of visual art.
- GorbachevyChase 7mo agoI’m OK with a Chinese model getting the W. It’s ultimately good for all of us.
- hn8726 7mo ago[flagged]
- lambda 7mo agoIt's a lighthearted, fun, visual benchmark that's not part of the standard benchmarks; and at least traditionally, it was not something that the labs trained on so it was something of a measure of how well the intelligence of the model generalized. Part of the idea of LLMs is that they pick up general knowledge and reasoning ability, beyond any tasks that they are specifically trained for, from the vast quantity of data that they are trained on. Of course, a while back there was a Gemini release that I believe specifically called out their ability to produce SVGs, for illustration and diagramming purposes. So it's not longer necessarily the case that the labs aren't training on generating SVGs, and in fact, there's a good chance that even if they're not doing so explicitly, the RLVR process might be generating tasks like that as there is more and more focus on frontend and design in the LLM space. So while they might not be specifically training for a pelican riding a bicycle, they may actually be training on SVG diagram quality.
- hamdouni 7mo agoMaybe this can help https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/
- hn8726 7mo agoIt doesn't, I get that it's _a_ benchmark. It's just not a good or insightful one, and having it posted so often on HN feels like low quality spam at this point
- VHRanger 7mo agoThe issue is that benchmarks that look insightful will end up being gamed by labs quickly (Goodharts law) The best LLM benchmarks test around the margins of those behaviors, tasks that are difficult and correlate with usefulness while being removed enough to stay unpolluted
- HarHarVeryFunny 7mo agoToo bad they didn't put equal effort into the pelican's legs and feet. Left leg paralyzed and not moving, and right ankle flipping around in alarming fashion!
- disiplus 7mo agowas part of the beta, its properly good model, in some sense i forgot that im not on opus or gpt. opus is still better. gpt is the one struggling for me. it has some niche in backend work but you can get the same with opus with skills, its lacking in almost all others.
- OtomotO 7mo agoFunny, for me Opus is struggling since about February. 4.7 made no difference, so for the first time in many moons I am cancelling my subscription.
- makingstuffs 7mo agoIt looks like a drunk pelican rolling downhill on its bicycle