4 ms·
Pelicans (thinking effort high, medium, low): https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Ff8820ce47db8749
by simonw 1mo ago
Pelicans (thinking effort high, medium, low): https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Ff8820ce47db87490734117e9be4984c3 https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - high cost 8.9742 cents
Here are the 3.7 pelicans for comparison: https://tools.simonwillison.net/markdown-svg-renderer.html?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2F6779a22d5e7bb6bdf29936f1600a5259 https://tools.simonwillison.net/markdown-svg-renderer.html?u... - high cost 8.4387 cents
(I think thinking level low is a regression on 3.8 compared to 3.7.)
- world2vec 1mo agoI mean no offense but these pelicans are a bit tiresome and a very meaningless benchmark. There's no real difference between any of these svgs across models and model versions anymore.
- simonw 1mo agoCongratulations, you're this thread's "pelicans are tiresome" comment - it's part of the Hacker News tradition at this point. (Next up is the comment saying that the labs are clearly training for the benchmark.)
- world2vec 1mo agoThe labs are clearly training for the benchmark.
- wongarsu 1mo agoIf everyone agreed with you, the comment would disappear near the bottom of the thread I like the benchmark. Yes, it's near saturation for SotA models, but still quite good to show where smaller models stand in relation to SotA In this instance, I see a great image, but consistently clipping mudguards (both in 3.8 flash and 3.7 flash)
- IshKebab 1mo ago> If everyone agreed with you, the comment would disappear near the bottom of the thread If only it were true that things that are tiresome are unpopular. But witness "6 7", "first post", ... remember the "in soviet Russia" jokes on Slashdot"? It seems like there are a subset of people that simply don't get tired of tiresome things.
- bitexploder 1mo agoIt is more fun than serious at this point. Don't overthink it :)
- anentropic 1mo agoit's a tradition
- onlyrealcuzzo 1mo agoThis is in comparison to Fable: > https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Facf6ab2516527d97f04b9f07d61a7cad https://tools.simonwillison.net/markdown-svg-renderer?url=ht... > Took just under 14 minutes to generate, and at 65927 output tokens cost me a hefty $3.30! So 50x cheaper - and how much faster?
- simonw 1mo agoThe Gemini models have openly trained for SVG output, apparently with a specialism on animals in forms of transport! https://twitter.com/JeffDean/status/2024525132266688757 https://twitter.com/JeffDean/status/2024525132266688757
- dieortin 1mo agoI don’t know if you’re joking, but I don’t see anything in the linked tweet which suggests that is the case
- simonw 1mo agoWatch the video. It's from then-Gemini-lead Jeff Dean and the video shows off an animated pelican riding a bicycle, a frog on a penny-farthing, a giraffe driving a tiny car, an ostrich on roller skates, a turtle kickflipping a skateboard, and a dachshund driving a stretch limousine.
- JacobAsmuth 1mo agoI'm confused. This doesn't mean they trained on it.
- simonw 1mo agoI didn't say trained on, I said "trained for SVG output". Gemini team members have publicly stated that they have trained for SVG: https://twitter.com/sunjiao123sun_/status/2024555516551372920 https://twitter.com/sunjiao123sun_/status/202455551655137292... > I’ve been developing the SVG generation capabilities for Gemini 3.1, and the complexity of the SVGs is stunning. > This allows UX designers to transcend pixel constraints and directly output structural, production-ready code!
- deleted 1mo ago[deleted]
- lern_too_spel 1mo agoThe fenders are a nice touch, but putting the fenders through the tires seems like a design flaw.
- jpadkins 1mo agoThe rendering of the gullet is very poor, because its both behind the handlebars but in front of the bike frame (impossible geometry). Surprising because gemini is usually pretty good on geo spatial skills. Edit: scrolled down to medium effort, its better but also has a weird clipping issue with the fish in the beak.
- neuronic 1mo agoLLMs are not intelligent and don't actually understand the concept of a bicycle. Parrots also don't understand human language but they're really good at pretending otherwise.
- jpadkins 1mo agoNeurons are not intelligent and don't actually understand the concept of a bicycle. LLM are just one component of the system we call "AI" used to generate the SVG. The parrot argument is tiresome because the LLM alone is not what enabled the recent take-off in practical intelligence we have seen in the last 12 months. The tooling, reasoning chains, harness to self evaluate the output, etc, etc are all needed to make the pelican. Just like the prefrontal cortex alone can't make a pelican with humans.
- hughw 1mo agoIt's about to squash a tiny baby pelican
- mrdependable 1mo agoWhy are the SVGs getting more detailed rather than just more correct than previous models?
- trentor 1mo agoBecause people tend to like fidelity more than correctness.
- aesthesia 1mo agoIt bugs me a little that "fidelity" has connotations other than "faithfulness to an original"---fidelity should be basically the same as correctness here!
- nonethewiser 1mo agoThat’s really not the highest fidelity interpretation of “fidelity.”
- aesthesia 1mo agoFair, but correctness should be a necessary component of fidelity.
- frumiousirc 1mo agoMy dictionary writes: "precision * accuracy = fidelity".
- neuronic 1mo agoIf correctness would matter anymore, people wouldn't be using LLMs in the first place.
- EugeneOZ 1mo agoImpressive pelicans!
- anigbrowl 1mo agoThese are becoming unreadable as the reasoning chains expand. I think you should consider reformatting them and either putting the image first or else folding the COT output.
- simonw 1mo agoYeah, putting reasoning in a details/summary is a good idea.
- deleted 1mo ago[deleted]