5 ms·
Pelican for Fable 5 on default settings is a clear improvement on Opus 4.8 Fable 5 default: https://gist.github.com/simonw/036bee5a703e7ec84e34efa97443828d htt
by simonw 5mo ago
Pelican for Fable 5 on default settings is a clear improvement on Opus 4.8
Fable 5 default: https://gist.github.com/simonw/036bee5a703e7ec84e34efa97443828d https://gist.github.com/simonw/036bee5a703e7ec84e34efa974438...
Opus 4.8 (the "max" one is closest to Fable): https://simonwillison.net/2026/May/28/claude-opus-4-8/#and-some-pelicans https://simonwillison.net/2026/May/28/claude-opus-4-8/#and-s...
Now here are the Fable pelicans for all five of the thinking effort levels - low, medium, high, xhigh, max: https://tools.simonwillison.net/markdown-svg-renderer#url=https%3A%2F%2Fgist.github.com%2Fsimonw%2F94fde31c34a0400c1d29f57e6a708e6b https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
Low used 25 input, 1,929 output - 9.67 cents: https://www.llm-prices.com/#it=25&ot=1929&sel=claude-fable-5 https://www.llm-prices.com/#it=25&ot=1929&sel=claude-fable-5
Max used 25 input, 14,430 output - 72.175 cents! https://www.llm-prices.com/#it=25&ot=14430&sel=claude-fable-5 https://www.llm-prices.com/#it=25&ot=14430&sel=claude-fable-...
- ealready_value 5mo agoThis is the reply I look for in all the new model announcements. Its fun to tell people that I judge models based on pelicans.
- pixel_popping 5mo agoThis is all we need, that moment the Pelican put the leg behind the frame, we are all doomed.
- chorkpop 5mo agoNow someone post the link about how it’s impossible for humans to draw a bike from memory.
- Atheros 5mo agohttps://link.springer.com/article/10.3758/BF03195929 https://link.springer.com/article/10.3758/BF03195929
- upcoming-sesame 5mo agoI also look for this reply because i like seeing the follow-up reply saying that this is not a benchmark anymore because labs have gotten it in their training data. that reply never failed to come it's basically a meme at this point
- mercacona 5mo agoWhy always sunny days?
- umeshunni 5mo agoPelicans hate biking in the rain (as do I).
- ethanlipson 5mo agoHow much money do you think they spent fine-tuning on pelican SVG generation?
- csomar 5mo agoProbably none. They probably have much better targets to optimize for than an SVG pelican or even SVGs in general.
- tarruda 5mo agoNot as much as Qwen, since apparently 3.6 35B surpassed Opus 4.7 https://x.com/simonw/status/2044830134885306701 https://x.com/simonw/status/2044830134885306701
- leecommamichael 5mo agoLooks like Fable constructed the "max" "looking" pelican of the previous model for the "xhigh" output token count of the previous model.
- sarreph 5mo agoI'm beginning to wonder how much of a useful metric the pelican is because surely the frontier labs must be training their models on pelican-artistry because of how well known your test is now?
- HaZeust 5mo agoI've seen this reply to Simon's benchmark for 2 years running now, and yet you still see improvements and objectively-bad results over time from new releases, even when I'm sure every frontier AI team has/had a person at least partially dedicated to better bicycle-pelican SVG outputs. Alas.
- sarreph 5mo agoI had intended to caveat that: I'm sure I'm not the first person to ask about this! > you still see improvements This is expected if they are training their models on it, right? > objectively-bad results Keen to learn when this has been the case, i.e. across version increments in major models.
- simonw 5mo agoI've written about this a couple of times, most notably here: https://simonwillison.net/2025/Nov/13/training-for-pelicans-riding-bicycles/ https://simonwillison.net/2025/Nov/13/training-for-pelicans-... I've been enjoying seeing how the quality of individual models differ based on the amount of reasoning effort you give them. If they were baking an a good pelican you wouldn't expect them to differ so much. (Google Gemini are the only lab that have very clearly paid attention to the quality of SVG animals-riding-vehicles, see their announcement for Gemini 3.1: https://twitter.com/JeffDean/status/2024525132266688757 https://twitter.com/JeffDean/status/2024525132266688757 )
- sarreph 5mo agoAmazing, thank you Simon! Look forward to reading.
- 5mo ago
- csomar 5mo agoWhere is the clear improvement on Fable 5? The tail is misplaced.
- redox99 5mo agoIt's interesting that they still get the head tube / handle bar part wrong.
- aarjaneiro 5mo agoOr the hands not being wings
- kylehotchkiss 5mo agoHow many barrels of oil are burned per pelican at Fable levels?
- ge96 5mo agoneed more Alex Moulton style bikes
- rkuska 5mo agoIs it possible to use the credits from subscription (https://support.claude.com/en/articles/15036540-use-the-claude-agent-sdk-with-your-claude-plan https://support.claude.com/en/articles/15036540-use-the-clau...) for fable?
- david_shi 5mo agothat's a great looking pelican
- 382hi 5mo agoI'm pretty sure they're optimizing the models around these sorts of tests.
- sempron64 5mo agoThe pelican has looked very same-y across all frontier models, same color bike, same camera angle, etc. I suspect this challenge is already too embedded in the training data to be a good signal when it succeeds, and maybe even when it fails in pathological ways mirroring existing AI pelicans on the internet.
- tripleee 5mo ago[flagged]
- jurgenaut23 5mo agoTrue that
- scrollaway 5mo agoDo you seriously have a dedicated “bad takes on AI” hn account?
- tripleee 5mo agoyeah, although I do combine it with "replies to snarky questions" for efficiency
- yreg 5mo agoI really don't understand what's interesting about this test and why is it always on top.
- makingstuffs 5mo agoI could be tripping but I’m sure that is very similar to the Deepseek one from not long ago. Clearly I am too lazy to go and find it for verification.
- jerryliu12 5mo agoPersonally feel like it could be more ambitious with what it creates.
- gavinray 5mo agoFable 5 xhigh actually looks the best to me.
- bergheim 5mo agoAnyone care about these pelicans that always come up anymore? Clearly at this point they are part of the training data. They even all look sort of ish the same. Daytime, colors,...
- 1attice 5mo agoWithout being mean, I encourage you to go look at some of simonw's writing on this topic, which he has addressed repeatedly (and IMO satisfactorily.) I know because I too had this initial take; however, upon analysis, it is not sound.
- bergheim 5mo agoI know he is an AI influencer that promotes his blog any chance he gets. I agree as well that he writes many interesting things.
- deleted 5mo ago[deleted]
- purple-leafy 5mo agoDo we need a pelican every single time a model is released? Beating a very dead horse. Fun at first, seems disingenuous now. A site funnel
- raffael_de 5mo agoI find it quite interesting that while the picture looks better the more advanced the model is, but apparently none so far "understands" that the pelicans legs are on both sides of the bike / top bar.
- LordDragonfang 5mo agoIf you scroll to the bottom of the Fable-5 by effort page, Max effort actually gets this correct! (Along with being the only one I've seen so far to make a bicycle frame that matches the shape of what most bikes on Google images look like)
- wasabi991011 5mo agoAnd the only one linked here that includes a bicycle chain!
- smusamashah 5mo agoCan you please compare the code generated by other similar quality pelicans by other models. Code in your first link (Fable 5 Default) looks minimal yet very good.
- lacoolj 5mo agodude, the max version looks like it's finally there. handle bar holding with wings, the left leg is behind the frame while the right is in front of it (correctly). well done anthropic.
- benatkin 5mo agoThe way they talked it up, having both legs on one side of the bike is like walking to the car wash
- ceroxylon 5mo agoYay, max level actually put one of the legs behind the frame!
- mer_mer 5mo agoIt's interesting that Gemini 3(.1?) Deep Think is still the best at this task and it's still not really generally available. Maybe Fable could match it at higher effort levels? https://simonwillison.net/2026/Feb/12/gemini-3-deep-think/ https://simonwillison.net/2026/Feb/12/gemini-3-deep-think/
- arthurcolle 5mo agomediocre pelican. very disappointing
- Reebz 5mo agoThe Max version gets more details right. The bike frame looks good, the chain, the wings are appropriately styled instead of “arms”, and the knee is bent, etc. Obviously we’re hitting marginal returns now, but I see differences.
- XCSme 5mo agoIt also does A LOT better, for my hamster test: https://aibenchy.com/showcase/?q=claude#showcase=6efb87c28e3152b2 https://aibenchy.com/showcase/?q=claude#showcase=6efb87c28e3...