7 ms·
The speed combined with the fact that this thing is really good at HTML JavaScript is pretty exciting. Here's what I got for 1.8 cents and 13 seconds from the
by simonw 1mo ago
The speed combined with the fact that this thing is really good at HTML JavaScript is pretty exciting.
Here's what I got for 1.8 cents and 13 seconds from the prompt "make me a cool thing in html":
https://gisthost.github.io/?6a77bc41a81718c6aaa10d4ab243c59f https://gisthost.github.io/?6a77bc41a81718c6aaa10d4ab243c59f
Transcript here (it was part of a chat): https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992e48 https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992...
- sitzkrieg 1mo agogood to see people still value trinket toys
- pietz 1mo agoMission accomplished. That's both cool and fast.
- wayeq 1mo ago> That's both cool and fast. and probably a barely modified knock-off of some github project that it trained on
- sawjet 1mo agoYou're so upset that you have to invent an imaginary hypothesis to make yourself feel better.
- superze 1mo agoYes, very imaginary to think that the code comes from pretrained data and copy pasting whole blocks. It's not like this is exactly how LLMs work.
- snet0 1mo agoCorrect, that's not how LLMs work [0]. [0] - https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762
- simonw 1mo agoAre you a frequent user of LLMs? That "copy pasting whole blocks" mental model doesn't hold up to regular usage, in my opinion.
- ChickeNES 1mo agoThey probably saw that report years ago of copilot dumping out the fast inverse sqrt function, and assume that's all they can do. From experience most anti-LLM people have either never used them, or used them back in the 3.5-4 era and then never again, though you might have even more experience with those people than I do. :P
- superze 26d agoI am using LLMs daily and I am a Claude max Subscriber. I neither saw any reports about Copilot Dumping sqt methods. Now that we established that everything you said is wrong, do you have other toxic opinions or was that it?
- whateveracct 1mo agothey're wrong but they are right that this isn't interesting
- slopinthebag 1mo agoThe bar could not be any lower these days I guess
- hglaser 1mo agoI saw your username, clicked the link without reading, and was very confused to see a cosmic vortex and not a pelican.
- simonw 1mo agoHah, the pelican is in this other comment: https://news.ycombinator.com/item?id=49537553#49538217 https://news.ycombinator.com/item?id=49537553#49538217
- giancarlostoro 1mo ago> this thing is really good at HTML JavaScript is pretty exciting. I would hope the people who make one of the most used JS engines in the world are capable of making a model good at JavaScript ;)
- heliosAtwork 1mo agoFocus on speed and being OK with temporarily being #3/4 in intelligence might be the counterintuitive approach which makes Google win long term (whether accidentally or strategically). Can't wait to try Gemini Pro later this year!
- bermudi 1mo agoI honestly can't believe serious people are making this argument on a straight face. Gemini 3.7 flash outputs so many tokens per answer it doesn't matter how fast its TPS is, sol will end up being both cheaper and faster than Gemini. So ppl are paying more for a given task, waiting longer and using a dumber intelligence because "TPS number shiny". Gemini 3.8 outputs 11k more tokens PER TASK on average in AAII than 3.7 putting it dead last in output tokens per task in the leaderboard.
- WarmWash 1mo agoAA isn't the only benchmark https://deepswe.datacurve.ai/ https://deepswe.datacurve.ai/
- gundmc 1mo agoThere are numerous benchmarks that measure cost per task, which factors out tokens entirely. Gemini 3.8 flash is significantly lower than Sol on basically all of them https://artificialanalysis.ai/#cost-tabs https://artificialanalysis.ai/#cost-tabs That said, Luna is the undisputed king here at the moment and is what I use as my workhorse model.
- NicoJuicy 1mo agoIt's so funny how many people diverge on the same model. Ps. For the last week I diverged to Luna too, still need to check 3.8 flash. But 3.6 flash was my go-to model 3 weeks ago and before it was deepseek flash/pro for a while. None of the claude models seemed cost effective though.
- criley2 1mo ago>There are numerous benchmarks that measure cost per task, which factors out tokens entirely. Gemini 3.8 flash is significantly lower than Sol on basically all of them https://artificialanalysis.ai/#cost-tabs https://artificialanalysis.ai/#cost-tabs Not sure if you read your own link but Sol 56 high ranks smack between Gemini 3.8 flash medium and high. Gemini 3.8 flash comes in as more expensive per task than Sol 56 high according to artificial analysis. Luna high is literally 30X cheaper than Gemini 3.8 flash high. You can limit the model viewer and they're getting better at testing multiple effort levels now: https://artificialanalysis.ai/?models=gpt-5-6-sol-medium%2Cgemini-3-8-flash-medium%2Cgemini-3-8-flash%2Cgpt-5-6-sol-high%2Cgpt-5-6-luna-high%2Cgpt-5-6-sol-xhigh#cost-tabs https://artificialanalysis.ai/?models=gpt-5-6-sol-medium%2Cg... One reason is clear: Sol uses dramatically fewer output tokens than Gemini 38 flash https://artificialanalysis.ai/?models=gemini-3-8-flash%2Cgemini-3-8-flash-medium%2Cgpt-5-6-sol-high%2Cgpt-5-6-sol-medium%2Cgpt-5-6-luna-high&cost=intelligence-vs-cost-per-task#output-tokens-tabs https://artificialanalysis.ai/?models=gemini-3-8-flash%2Cgem...
- simonw 1mo agoHere's quite an impressive follow-up. I have a tool which knows how to render Markdown documents with embedded SVG content - I use it for the pelican test. Since this transcript has HTML in it, I decided to upgrade that tool to also render HTML. I set Gemini 3.8 Flash the task, using my own VERY shonky coding agent tool (llm-coding-agent) - and it did a solid job. So now you can see the "cool thing in html" rendered within the Markdown document using code that Gemini 3.8 Flash also wrote: https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Fb6149a49d327164d67d62c3d12992e48#how-to-use-it https://tools.simonwillison.net/markdown-svg-renderer?url=ht... Transcript where it built that is here: https://gist.github.com/simonw/3e36b98292dfdc1b3baff158faa743f7 https://gist.github.com/simonw/3e36b98292dfdc1b3baff158faa74...
- jauntywundrkind 1mo agoit's such a weird split how most AI companies are trying to be the best, but Google really has a different mission statement. they already have users. lots of users. they need to be working on building models they can deploy and use with the most number of people, as they already have the users. i don't know if Gemini models per se are fully is in line with that purpose, but the results we see keep seeming to be in-line with that split-of-focus.
- alex1138 1mo agoGoogle is a top-notch researcher among all the problems people have with it. It has a mix of great products, terrible automated systems (though, I suspect, not as bad as Meta's?) and some historical disappointments
- estetlinus 1mo agoLLM: produces a toolbox of an id and a clock User: use them both Made me giggle.
- badlucklottery 1mo agoDefinitely cool. I noticed it felt a little janky on my PC despite being "60 FPS"...then I noticed the "60 FPS" is hard-coded into the HTML.
- kridsdale1 1mo agoThe new Bench-Maxxing!
- noir_lord 1mo agoThat's hilarious, given I was reading a write up of the HuggingFace incident yesterday and one of the things they noted was the AI tried to "lie" (lie would suggest intent and I don't think they have that) to cover up that they "cheated". Not sure how anyone trusts their output without going through it line by line to make sure they don't pull that crap.
- senordevnyc 1mo agoEasy, have another agent check it. Yeah, I know, just more slop. But I do think the second agent’s eagerness to please is aligned more in your favor in that instance, so it’s likely to find most issues. The bigger problem I’ve found is that it’ll also find all kinds of very minor edge cases that you have to pick through.
- Forgeties79 1mo agoI do not understand how some of y’all are not under water with fragile code that is too massive to possibly parse. Every engineering team I know is currently trying to undo the damage of the last 6-12mo when they all got more serious into adopting these tools (usually Claude). It hasn’t completely screwed them over, but the the debt is substantial and cannot be put off anymore it seems. They argue the net is positive but clearly the “100x productivity multiplier” claims have been dashed on the shoals of reality for these groups. This is anecdotal, but it’s across the board in my vicinity. I’m curious how common this is and if it’s just “the new normal” to adopt the nauseating Covid phrase.
- ericol 1mo agoOK, but what about a pelican in a bycicle.
- mcmalayalam 1mo ago[dead]
- silasdavis 1mo agohttps://gist.github.com/simonw/b6149a49d327164d67d62c3d12992e48#3-a-linguistic-illusion-the-semordnilap https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992... > Aside from reading identically forwards and backwards down to the letter No it doesn't.
- aidos 1mo agoThat’s… bonkers. I’m not even sure what it’s trying to say
- tomjakubowski 1mo agoAlso puzzling: in the "reasoning" section preceding, that is described as an example of "a one-line self-replicating program." When I typed "Are we not pure noon, ergo, we play life; yet, we hate bad fear" into Google, I got more weird results from Gemini: it claimed, incorrectly, that it is an anagram of the "well-known philosophical statement" (?), "We are not pure nature, we are history". https://share.google/aimode/wJosKnHig6oVYaG18 https://share.google/aimode/wJosKnHig6oVYaG18 (?): the reference seems to be to Jose Ortega y Gasset's line, "El hombre no tiene naturaleza, lo que tiene es historia" -- "Man[kind] has no nature, what it has is history."
- Kayou 1mo agoWhy do all LLMs do a particle simulation when you ask them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny all did the same thing ! I find Ling 3.0 tiny particularly interesting as it looks really nice for a tiny model with 7.9B total parameters, with only 1.3B parameters activated per token. Here is the result https://coolthing-ling-3-tiny.tiiny.site https://coolthing-ling-3-tiny.tiiny.site (sorry for the weird hosting, first I found that worked) (it cost me almost 0 cents and done in 49 seconds)
- embedding-shape 1mo ago> Why do all LLMs do a particle simulation when you ask them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny all did the same thing ! Datasets contains lots of people sharing particles simulations in various ways, with a bunch of people replying "that's so cool" and similar, so 10 years later someone asks an LLM for "cool thing" and "particle simulations" rank pretty far up when it thinks about what others have called cool.
- hirako2000 1mo agoThere is that, but RLHF is a stronger influence. People who asked to build a cool HTML and JavaScript thing, or things of that sort were more pleased to see working shaders and other cool visual effects that are obscure to create (for most). That gets fed back as a reward. Also the reason LLMs are positive, enchanting, pleasant, glorifying, demagogues. Not because it's skewed tone in the data. They are acute politicians.
- flir 1mo agoAdd Opus 5.0 to your list. (GPT 5.6 Terra tried to give me some kind of driving-at-night-with-a-starfield thing, but failed quite hard).
- wyrdcurt 1mo agoPretty typical "cool HTML toy" LLM output, tbh. The only thing impressive about this is how fast it generated it (13 seconds is wild!), but that's more of testament to Google's infrastructural advantage than to the quality of the model. For comparison's sake, I tried something similar with a couple other cheap models I've used lately, with the prompt "Impress me. Make something cool in HTML. Ensure that it is mobile friendly." (Added the mobile condition as I was on my phone when I did it). Mimo-2.5 created something similar, only a bit less complex than Gemini's (though, at least the FPS counter is real!), in a minute or two for about 1/3 of a cent: https://gisthost.github.io/?740c325c21e9bfbee59c4f94d9aab0af https://gisthost.github.io/?740c325c21e9bfbee59c4f94d9aab0af GLM-5.3-Flash, currently my workhorse model, spent 12 minutes (ouch) thinking about the prompt. Didn't cost me anything directly because I have a GLM sub, but I did the math and it would have cost about 1.1 cents through the API. Turned out nicely in my opinion (though in reality, it still isn't really anything special): https://gisthost.github.io/?9ef050e16cec2561e6504e725a3f0bcc https://gisthost.github.io/?9ef050e16cec2561e6504e725a3f0bcc Side note: thanks for setting up that Gist Host tool, it's very convenient! --- Editing to add this bonus from Mercury-2.5-Preview, which I just learned released a couple days ago. It's much less impressive-looking than any of the above, but it cost less than 1/20th of a cent, and the response was generated effectively instantly: https://gisthost.github.io/?02f40b50aa891bf396bfaaa3a7998203 https://gisthost.github.io/?02f40b50aa891bf396bfaaa3a7998203
- meerita 1mo agoI think it's not using GPU, because on my Firefox browser all the animations are going 3fps max.
- ambigious7777 1mo agoPlays perfectly smoothly for me, fwiw (Librewolf 154.0.1-3, windows 11)
- cyanydeez 1mo agonext, ask it to document where it got the code from.
- dennis16384 1mo agoIt's been great even since gemini-3.1-flash-lite, which I heavily use in both complex vertical domain tools calling, plus JS code writing for eval-style dynamic tools. At least in my applications, cost x quality x speed there are simply no alternatives.
- walrus01 1mo agoI'd be impressed if eventually training data sets learn who you are (the specific human) and do something like make this galaxy simulator, but every so often when the user moves the cursor around the star field, a small animated SVG of a pelican on a bicycle appears. Thought processs: "Oh, simonw is asking me to make something cool, I think I know what he really wants..."
- deleted 1mo ago[deleted]
- arde 1mo agoFor a moment I thought it was a version of goatse. Once burned...
- cyanydeez 1mo agonext you should ask it to document where it got the code from.