4 ms·
This doesn't have API access yet, but OpenAI seem to approve of the Codex API backdoor used by OpenClaw these days... https://twitter.com/steipete/status/204677
by simonw 7mo ago
This doesn't have API access yet, but OpenAI seem to approve of the Codex API backdoor used by OpenClaw these days... https://twitter.com/steipete/status/2046775849769148838 https://twitter.com/steipete/status/2046775849769148838 and https://twitter.com/romainhuet/status/2038699202834841962 https://twitter.com/romainhuet/status/2038699202834841962
And that backdoor API has GPT-5.5.
So here's a pelican: https://simonwillison.net/2026/Apr/23/gpt-5-5/#and-some-pelicans https://simonwillison.net/2026/Apr/23/gpt-5-5/#and-some-peli...
I used this new plugin for LLM: https://github.com/simonw/llm-openai-via-codex https://github.com/simonw/llm-openai-via-codex
UPDATE: I got a much better pelican by setting the reasoning effort to xhigh: https://gist.github.com/simonw/a6168e4165a258e4d664aeae8e602cc5?permalink_comment_id=6115759#gistcomment-6115759 https://gist.github.com/simonw/a6168e4165a258e4d664aeae8e602...
- DrProtic 7mo agoThat pelican you posted yesterday from a local model looks nicer than this one. Edit: this one has crossed legs lol
- BeetleB 7mo agoIt really needs to pee.
- droidjj 7mo agoIt's... like no pelican I've ever seen before.
- hagbard_c 7mo agoYou've never seen pelicans riding bicycles either so maybe these are just representations of those specific subgroups of pelicans which are capable of riding them. Normal pelicans would not feel the need to ride bikes since they can fly, these special pelicans mostly seem to lack the equipment needed to do that which might be part of the reason they evolved to ride two-wheeled pedal-propelled vehicles.
- XCSme 7mo agoIs this direct API usage allowed by their terms? I remember Anthropic really not liking such usage.
- simonw 7mo agoApparently it's fine: https://twitter.com/romainhuet/status/2038699202834841962 https://twitter.com/romainhuet/status/2038699202834841962
- andriy_koval 7mo agowhat is your setup for drawing pelican? Do you ask model to check generated image, find issues and iterate over it which would demonstrate models real abilities?
- simonw 7mo agoIt's generally one-shot-only - whatever comes out the first time is what I go with. I've been contemplating a more fair version where each model gets 3-5 attempts and then can select which rendered image is "best".
- andriy_koval 7mo agoI think it will make results way better and more representative of model abilities..
- simonw 7mo agoIt would... but the test is inherently silly, so I'm still not sure if it's worth me investing that extra effort in it.
- irthomasthomas 7mo agoTry llm-consortium with --judging-method rank
- deflator 7mo agoHmm. Any idea why it's so much worse than the other ones you have posted lately? Even the open weight local models were much better, like the Qwen one you posted yesterday.
- deleted 7mo ago[deleted]
- simonw 7mo agoThe xhigh one was better, but clearly OpenAI have not been focusing their training efforts on SVG illustrations of animals riding modes of transport!
- irthomasthomas 7mo agoIt beats opus-4.7 but looks like open models actually have the lead here.
- postalcoder 7mo agoI made pelicans at different thinking efforts: https://hcker.news/pelican-low.svg https://hcker.news/pelican-low.svg https://hcker.news/pelican-medium.svg https://hcker.news/pelican-medium.svg https://hcker.news/pelican-high.svg https://hcker.news/pelican-high.svg https://hcker.news/pelican-xhigh.svg https://hcker.news/pelican-xhigh.svg Someone needs to make a pelican arena, I have no idea if these are considered good or not.
- deflator 7mo agoThey are not good, and they seem to get worse as you increased effort. Weird
- postalcoder 7mo agoYeah. I've always loosely correlated pelican quality with big model smell but I'm not picking that up here. I thought this was supposed to be spud? Weird indeed.
- throw310822 7mo agoNo but I can sense the movement, I think it's already reached the level of intelligence that draws it towards futurism or cubism /s
- seanw444 7mo agoCan someone explain how we arrived at the pelican test? Was there some actual theory behind why it's difficult to produce? Or did someone just think it up, discover it was consistently difficult, and now we just all know it's a good test?
- Gander5739 7mo agohttps://simonwillison.net/2025/Jun/6/six-months-in-llms/ https://simonwillison.net/2025/Jun/6/six-months-in-llms/
- redox99 7mo agoIt all began with a Microsoft researcher showing a unicorn drawn in tikz using GPT4. It was an example of something so outrageous that there was no way it existed in the training data. And that's back when models were not multimodal. Nowadays I think it's pretty silly, because there's surely SVG drawing training data and some effort from the researchers put onto this task. It's not a showcase of emergent properties.
- sjdv1982 7mo agoAt some point, OpenAI is going to cheat and hardcode a pelican on a bicycle into the model. 3D modelling has Suzanne and the teapot; LLMs will have the pelican.
- SkyBelow 7mo agoWait, I thought we were onto racoons on e-scooters to avoid (some of) the issues with Goodhart's Law coming into play.
- simonw 7mo agoI fall back to possums on e-scooters if the pelican looks too good to be true. These aren't good enough for me to suspect any fowl play.
- rolymath 7mo agoExciting. Another Pelican post.
- refulgentis 7mo agoIt's silly and a joke and a surprisingly good benchmark and don't take it seriously but don't take not taking it seriously seriously and if it's too good we use another prompt and there's obvious ways to better it and it's not worth doing because it's not serious and if you say anything at all about the thread it's off-topic so you're doing exactly what you're complaining about and it's a personal attack from the fun police. Only coherent move at this point: hit the minus button immediately. There's never anything about the model in the thread other than simon's post.
- simonw 7mo agoSee if you can spot what's interesting and unique about this one. I've been trying to put more than just a pelican in there, partly as a nod to people who are getting bored of them.
- dakolli 7mo agoYou know they are 1000% training these models to draw pelicans, this hasn't been a valid benchmark for 6 months +
- simonw 7mo agoOpenAI must be very bad at training models to draw pelicans (and bicycles) then.
- Legend2440 7mo agoSkeptism is out of control these days, any time an LLM does something cool it must have been cheating.
- dakolli 6mo agothey legitimately suck at everything they don't have concrete examples to copy from.
- gpm 7mo agoI for one delight in bicycles where neither wheel can turn! It continues to amaze me that these models that definitely know what bicycle geometry actually looks like somewhere in their weights produces such implausibly bad geometry. Also mildly interesting, and generally consistent with my experience with LLMs, that it produced the same obvious geometry issue both times.
- lxgr 7mo ago> It continues to amaze me that these models that definitely know what bicycle geometry actually looks like somewhere in their weights produces such implausibly bad geometry. I feel like the main problem for the models is that they can't actually look at the visual output produced by their SVG and iterate. I'm almost willing to bet that if they could, they'd absolutely nail it at this point. Imagine designing an SVG yourself without being able to ever look outside the XML editor!
- gpm 7mo ago> Imagine designing an SVG yourself without being able to ever look outside the XML editor! I honestly think I could do much better on the bicycle without looking at the output (with some assistance for SVG syntax which I definitely don't know), just as someone who rides them and generally knows what the parts are. I'd do worse at the pelicans though.
- GistNoesis 7mo agoIsn't it awful ? After 5.5 versions it still can't draw a basic bike frame. How is the front wheel supposed to turn sideways ?
- simonw 7mo agoYeah, the bike frame is the thing I always look at first - it's still reasonably rare for a model to draw that correctly, although Qwen 3.6 and Gemini Pro 3.1 do that well now.
- loa_in_ 7mo agoThe distinction is that it's not drawing. It's generating an SVG document containing descriptors of the shapes.
- jetrink 7mo agoI feel like if I attempted this, the bike frame would look fine and everything else would be completely unrecognizable. After all, a basic bike frame is just straight lines arranged in a fairly simple shape. It's really surprising that models find it so difficult, but they can make a pelican with panache.
- fragmede 7mo agoMy question is, as a human, how well would you or I do under the same conditions? Which is to say, I could do a much better job in inkscape with Google images to back me up, but if I was blindly shitting vectors into an XML file that I can't render to see the results of, I'm not even going to get the triangles for the frame to line up, so this pelican is very impressive!
- necubi 7mo agoHumans are also famously bad at drawing bicycles from memory https://www.gianlucagimini.it/portfolio-item/velocipedia/ https://www.gianlucagimini.it/portfolio-item/velocipedia/
- deleted 7mo ago
- noonething 7mo agoThank you for doing all this. It's appreciated.
- i_love_retros 7mo agoYou do realise they are doing it for self promotion right?
- simonw 7mo agoI mean, yeah. "Person who spends time publishing content online is doing it for self promotion" doesn't seem particularly notable to me. 24 years of self promotion and counting!
- fc417fc802 7mo agoI am always outraged when youtube creators ask me to like and subscribe. /s
- i_love_retros 7mo agoNot the same at all. For that to happen you would have to explicitly visit their channel (forgive incorrect terminology, I don't use youtube). If someone kept posting on hackernews asking you to subscribe I hope you wouldn't appreciate it. swillison is spamming a communal public feed with self promotional comments about vibe coding, quite obviously because they, like the rest of us, are panicking about not having a career in a few years.
- simonw 7mo agoThe more time I spend actually working with these tools the less I fear for my future career. Building software remains really hard. Most people are not going to be able to produce production quality software systems, no matter how good the AI tooling gets.
- Schlagbohrer 7mo agoThat's amazing that the default did that much in just 39 "reasoning tokens" (no idea what a reasoning token is but that's still shockingly few tokens)
- erdaniels 7mo agoIf you don't know what a reasoning token is, then how can 39 be considered shockingly few?
- Culonavirus 7mo agoIt's less than 67, duh.
- tclancy 7mo agoNot during peak hours.
- singingtoday 7mo agoThank you for continuing to post these! Very interesting benchmark.
- stingraycharles 7mo agoOpenAI hired the guy behind OpenClaw, so it makes sense that they’re more lenient towards its usage.
- thierrydamiba 7mo agoThey basically bought OpenClaw right?
- takethebus 7mo agoI believe the technical term is "acquihire"
- deleted 7mo ago[deleted]
- matt3210 7mo agoThe pelican doesn’t really matter anymore since models are tuned for it knowing people will ask.
- simonw 7mo agoThey suck at tuning for it.
- zerop 7mo agoSo pelican must have become the mandatory test case to pass for all model providers before launch.
- mannanj 7mo agoDoes OpenAI actually act open for once here, and allow using their model via a subscription over Anthrophic banning use in Openclaw?
- simonw 7mo agoThat's what they said on Twitter.