5 ms·
If Chrome has the #optimization-guide-on-device-model and #prompt-api-for-gemini-nano flags enabled, either because it's part of some Origin Trial / Early Stabl
by scriptsmith 5mo ago
If Chrome has the #optimization-guide-on-device-model and #prompt-api-for-gemini-nano flags enabled, either because it's part of some Origin Trial / Early Stable Release or something, then web pages will have access to the new Prompt API which allows any webpage to initiate the (one-time) download of the ~2.7 GiB CPU or ~4.0 GiB GPU model using LanguageModel.create()
https://developer.chrome.com/docs/ai/prompt-api https://developer.chrome.com/docs/ai/prompt-api
When Chrome 148 releases tomorrow, this will be the default behaviour on desktop.
To download, it should check for 22 GiB free disk space on the volume where your Chrome data dir is, and at least double the model size of free space in your tmp dir.
- tobylane 5mo agoThose two (and more) exist in chrome://flags in Chrome 147. I'm disabling them now, with the expectation that will prevent the new default. One option I'm leaving as default is "Use LiteRT-LM runtime for on-device model service inference." Any comment on that?
- scriptsmith 5mo agoThose flags will exist already, but will default to enabled in 148. That other flag is for using a different open-source inference engine to the (from what I can tell) closed-source one that's used by default.
- RaiausderDose 5mo agoI'm on Chrome 147 too and disabled: "optimization-guide-on-device-model" - Enables optimization guide on device "prompt-api-for-gemini-nano" - Prompt API for Gemini Nano - Prompt API for Gemini Nano with Multimodal Input and deleted weights.bin and the 2025.x folder in "OptGuideOnDeviceModel" Will report if Chrome 148 downloads the model again.
- phs318u 5mo agoIf you touch those files into existence and chown to root and chmod to 0, it shouldn’t be able to ever overwrite them right?
- RaiausderDose 5mo agoyeah, should work. Will try readonly on windows too. Now I can't see it anymore, but shouldn't the model be under chrome://on-device-internals/ -> model-status? Maybe you can uninstall there too.
- pmontra 5mo agoI'm on my phone now so I can't check if something has changed, but what you want to protect from change is the directory, not the files. A file can be deleted and created again if the process can write the directory.
- sethops1 5mo agoYou want to use chattr +i (make the empty file immutable)
- Markoff 5mo agothanks, went to flags in Vivaldi and just in case disabled all flags containing "gemini" and first five results for "model"
- beaugunderson 5mo agomaybe I was on the wrong side of the early release but I’ve deleted this model many times in the last year. I’ve had it for at least 12 months.
- RaiausderDose 5mo agoit downloaded the model again...
- dpoloncsak 5mo ago[dead]
- wuschel 5mo agoIt is a small model, so what utility can I / Google expect from it? What is the on-board model used for?
- scriptsmith 5mo agoIt's based on Gemma 3n, and it's not the best. I find it works fine for simple classification, translation, interpretation of images & audio. It can write longer prose, but it's pretty bad. It can also write text in the format of a JSON schema or regexp for anything you might want to do with structured data.
- Wowfunhappy 5mo agoI wonder why they’re using Gemma 3 and not Gemma 4?
- andy_ppp 5mo agoIt'll probably update to that without telling you at some point.
- scriptsmith 5mo agoGoogle has been trialling the Prompt API in chrome for the over a year, so before Gemma 4 existed. But they are indicating they'll move to Gemma 4: https://groups.google.com/a/chromium.org/g/blink-dev/c/iR6R7-nQeHI/m/AM0yj_xTBgAJ https://groups.google.com/a/chromium.org/g/blink-dev/c/iR6R7...
- dotancohen 5mo agoSo that the big news in non-tech news sites will be the update. Thus ensuring that this is received in a positive light.
- 2ndorderthought 5mo agoIt's not a very good small model to be honest. That said, you might be surprised to learn that some of the models from 3b-9b could probably replace 80% of the things nonvibe coders use chatgpt for. Its a good idea to run small models locally if your computer can host them for privacy and cash saving reasons. But how can you trust Google to autoinstall one on your machine in 2026? I just couldn't do it.
- maxloh 5mo agoThe more severe problem is that Google installs model weight files on a per-user basis, meaning Chrome occupies 4 more GB of space for every OS user on your device.
- emegeve83 5mo agoFor every profile.
- bityard 5mo agoThe company I work at has several environments and hundreds of VDI users in each environment. Chrome is the default browser in all of them. By my rough napkin math, this one small change by Google will eat up at least 15 terabytes of new disk space in total. (I sure hope we are using deduplication at the physical storage layer...)
- throwway120385 5mo agoIt's fine. Network and disk space are free, right?
- charcircuit 5mo agoCompared to human labor it is.
- account42 5mo agoOnly because those who can save on the labor are not paying for the increased resource use in the first place.
- Pay08 5mo agoI certainly hope you don't automatically update.
- TheRealDunkirk 5mo ago
- 21asdffdsa12 5mo agoFirst the tabs came for the RAM and i did not protest, for i had plenty. Then they came for the chip and i did not protest, for it was dark silcon anyway. Then they came for the HDD.
- underlipton 5mo agoTold ya.
- doctorpangloss 5mo agoOkay, but the browser is basically the computer for most people.
- oaiey 5mo agoAnd then they made the ram and ssd so expensive :)
- bearjaws 5mo agoI am curious if it reuses the LLM across all tabs, hard to imagine most machines can boot up 1-2 of any 4gb model unless its a more powerful system.
- sheept 5mo agoYou can already trigger a 2 GB model download with the Summarizer API[0], which is already shipped in Chrome. Summarizer.create() [0]: https://developer.chrome.com/docs/ai/summarizer-api#model-download https://developer.chrome.com/docs/ai/summarizer-api#model-do... I think this is a distinct model from the Prompt API, since the other shipped AI APIs use fine tuned models.
- crumpled 5mo agoSo now we're up to 6 GB
- entropicdrifter 5mo agoPer user
- rafram 5mo agoBoth of them say they use Gemini Nano.
- deleted 5mo ago[deleted]
- BergAndCo 5mo ago[dead]
- jimmaswell 5mo agoThis sounds perfectly reasonable. No objection from me.
- codethief 5mo agoNext step: Invoke the prompt API from within online ads and run a "p2p" AI inference provider which forwards incoming LLM queries to website visitors. :-)
- ddtaylor 5mo agoThe problem is that some of us are still on connections that charge per GB in rural areas. Here in Montana it's very common to pay about $0.25 per GB regardless of how much you use, so this is a $1 additional cost per desktop device. Places like public school districts have hundreds of computers and this will be somewhat significant for them.
- McGlockenshire 5mo agoGoogle's updater service also currently ignores the windows 11 metered connection hint. It will gladly download that model over your cell connection even if you have a data cap. This is infuriating behavior. Silicon Valley must wake up and understand the entire world does not live like them.
- bjelkeman-again 5mo agoThey live in a bubble and not a lot of the surrounding world makes it in to them. I know it is hyperbolic, but I lived there for a while and I stand by that opinion.
- marklubi 5mo agoI was thinking a similar thing. Many of our customers have purpose use computers that rarely see physical infrastructure internet, but need a modern browser (many chose Chrome on their own, we never recommended it). They're going to get blasted with cellular data charges when they fire up their computer in the field.
- d3Xt3r 5mo agoSo my understanding of that is that the download happens only when sites call the Prompt API right? Because my Chrome stable has been updated to v148 now, and I don't see any AI models in my user profile folder. My profile size is only 328 MB, with the Code Cache subfolder occupying the most space (135 MB).
- scriptsmith 5mo agoIn my understanding, yes. I wrote a blog post about some of the internals here: https://news.ycombinator.com/item?id=48028662 https://news.ycombinator.com/item?id=48028662
- Twirrim 5mo agoSearching about:flags for model comes up with a whole bunch: #omnibox-ml-url-scoring-model #omnibox-on-device-tail-suggestions #optimization-guide-on-device-model #text-safety-classifier #prompt-api-for-gemini-nano #writer-api-for-gemini-nano #rewriter-api-for-gemini-nano #proofreader-api-for-gemini-nano #summarizer-api-for-gemini-nano #on-device-model-litert-lm-backend Then around gemini but not caught by the search for models: #skills (maybe? I think this is implied by "gemini in chrome"?) edit: I don't see a carte blanch AI disabling option. As much as I dislike Mozilla's growing obsession with AI, at least they give me a top level option to disable all AI stuff. I only keep Chrome around for occasional testing reasons.
- jadbox 5mo agoI believe webpages that use the API must request from the user via a system permissions dialogue to aces the prompt API, according the docs a few months ago.
- scriptsmith 5mo agoIt can only be called after the user has interacted with the page, but there's no dialogue from the browser https://developer.chrome.com/docs/ai/get-started#user-activation https://developer.chrome.com/docs/ai/get-started#user-activa...
- scriptsmith 5mo agoI wrote a more detailed blog post here: https://news.ycombinator.com/item?id=48028662 https://news.ycombinator.com/item?id=48028662
- madduci 5mo agoDo you know if also Chromium has thesenfkags enabled?
- scriptsmith 5mo agoDepends on where you get it. By default the flags will be enabled, but some packagers may choose to disable them. I haven't seen a major distro release chromium 148 yet. Weirdly though, chromium won't be able to actually use the model even though it can download it, because the inference engine is a closed-source blob. https://adsm.dev/posts/prompt-api/#which-browsers-support-the-api https://adsm.dev/posts/prompt-api/#which-browsers-support-th...
- Vinnl 5mo agoAlso note the Mozilla standards position on this API: https://github.com/mozilla/standards-positions/issues/1213#issuecomment-4347988313 https://github.com/mozilla/standards-positions/issues/1213#i... Or this summary on its status: > Mozilla: Opposed > WebKit: Opposed > Microsoft: Several concerns > W3C TAG: Several concerns > Developers: Mostly negative From https://mastodon.social/@jaffathecake/116527007495775507 https://mastodon.social/@jaffathecake/116527007495775507
- arendtio 5mo ago/dev/mapper/vg_system-arch 207G 192G 4,7G 98% / Just don't keep free space around :-D
- hzwanip 5mo agoI think it's great, LFG Chromium OSS