3 ms·
I wrote about that possibility here: https://simonwillison.net/2025/Nov/13/training-for-pelicans-riding-bicycles/ https://simonwillison.net/2025/Nov/13/training
by simonw 11mo ago
I wrote about that possibility here: https://simonwillison.net/2025/Nov/13/training-for-pelicans-riding-bicycles/ https://simonwillison.net/2025/Nov/13/training-for-pelicans-...
- th0ma5 11mo ago[flagged]
- dugidugout 11mo agoCondescending and disrespectful to whom? Everybody wholsale? This doesnt seem reasonable? Please elaborate.
- th0ma5 11mo agoNo, when did I say that?
- deleted 11mo ago[deleted]
- dugidugout 11mo agoIt isn't clear what you said. You asserted a pattern of conduct on the user simonw: > I think constantly replying to everybody with some link which doesn't address their concerns Then claimed that conduct was: > condescending and disrespectful. I am asking you to elaborate to whom simonw is condescending and disrespecting. I don't see how it follows.
- bravetraveler 11mo agoNot sure if I'd use the same descriptions so pointedly, but I can see what they mean. It's perfectly fine to link for convenience, but it does feel a little disrespectful/SEO-y to not 'continue the conversation'. A summary in the very least, how exactly it pertains. Sell us. In a sense, link-dropping [alone] is saying: "go read this and establish my rhetorical/social position, I'm done here" Imagine meeting an author/producer/whatever you liked. You'd want to talk about their work, how they created it, the impact it had, and so on. Now imagine if they did that... or if they waved their hand vaguely at a catalog.
- tomrod 11mo agoHell, I would consider myself graced that simonw, yes, THAT simonw, the LLM whisperer, took time out of his busy schedule to send me to a discussion I might have expressed interest in.
- bravetraveler 11mo ago> send me to a discussion I might have expressed interest in No, no, remember? Points to the blog you were already reading! Working diligently to build a brand: podcast, paid newsletter, the works.
- tomrod 11mo agoI wasn't speaking to this interaction, and my point is genuine. Simonw has done fantastic work in the LLM space
- bravetraveler 11mo ago... and my point remains: he's fine. Could be better. If he does grace us, he can choose to bait the hook more effectively. Or not. The stakes are silly-low. This interaction is, effectively, a link dropped with an announcement of an announcement. For what has already occurred. Over-fitting, training? You don't say. If I wanted to be more of an ass, I'd look to argue about hype generation. But I don't, I appreciate any honest effort, which I believe for Simon.
- simonw 11mo agoI've genuinely been answering the question "what if the labs are training on your pelican benchmark" 3-4 times a week for several months at this point. I wrote that piece precisely so I didn't have to copy and paste the same arguments into dozens of different conversations.
- bravetraveler 11mo agoOh, no. Does this policing job pay well? /s Seriously: less is more, trust the process, any number of platitudes work here. Who are you defending against? Readers, right? You wrote your thing, defended it with more of the thing. It'll permeate. Or it won't. Does it matter? You could be done, nothing is making you defend this (sorry) asinine benchmark across the internet. Not trying to (m|y)uck your yum, or whatever. Remember, I did say linking for convenience is fine. We're belaboring the worst reading in comments. Inconsequential, unnecessary heartburn. Link the blog posts together and call it good enough.
- vanschelven 11mo agoWhatever you think Jimmc414's _concerns_ are (they merely state a possibility) Simon enumerates a number of concerns in the linked article, and then addresses those. So I'm not sure why you think this is so.
- vnvnff 11mo agoIt's a pattern: https://news.ycombinator.com/item?id=44725190 https://news.ycombinator.com/item?id=44725190
- armcat 11mo agoHi Simon! Love your work! Our of curiosity - how many pelican-cycling samples do you produce. Curious about the variance here. Thanks!
- simonw 11mo agoI've lost count, but there are 85 posts with that tag here: https://simonwillison.net/tags/pelican-riding-a-bicycle/ https://simonwillison.net/tags/pelican-riding-a-bicycle/ I need to extract them all into a formal collection.
- jgalt212 11mo agoAiden is perhaps misinformed. From a Bing search performed just now. > Yes, I am familiar with the "pelican riding a bicycle" SVG generation test. It is a benchmark for evaluating the ability of AI models, particularly large language models (LLMs) and multi-modal systems, to generate original, high-quality SVG vector graphics based on a deliberately unusual and complex prompt. The benchmark was popularized by Simon Willison, who selected the prompt because:
- 100721 11mo agoWeb search-based RAG is very different from having something embedded in a model's training data, though.
- jgalt212 11mo agoChatGPT website gives a similar answer. Are they running RAG, or the model? > Yes — I’m familiar with the “pelican riding a bicycle” SVG generation test. > It’s become a kind of informal benchmark people use when evaluating whether an image-generation or SVG-generation model can: ...
- Barbing 11mo agoRunnin’ confabulations: >Yes — the “hamster driving a car” prompt is a well-known informal test … >…that’s a well-known informal test people use…(a mole-rat holding or playing a guitar). Try any plausible concept. Get sillier and it’s trained to talk about it being nonsense. The output still claims it’s a real test, just a real “nonsense” test.