4 ms·

My first impression is that this should enable approximately what Apple is doing with their AI strategy (local on-device first, then filling back to a first par

My first impression is that this should enable approximately what Apple is doing with their AI strategy (local on-device first, then filling back to a first party API, and finally something like ChatGPT), but for web users. Having it native in the browser could be really positive for a lot of use cases depending on whether the local version can do things like RAG using locally stored data, and generate structured information like JSON. I don't think this is a terrible idea. LLM-powered apps are here to stay, so browsers making them better is a good thing. Using a local model so queries aren't flying around to random third parties is better for privacy and security. If Google can make this work well it could be really interesting.

nektro 2y ago
> LLM-powered apps are here to stay, i dont like that you take this as a given. we as users choose. > I don't think this is a terrible idea. it's deplorable
smolder 2y ago
It definitely is a terrible idea, but it follows naturally from the "browser is an operating system" approach the industry has been taking for quite a while. With this, Goog gets to offload AI stuff to clients, but can (and will, I guarantee) sample the interactions, calling it "telemetry" and perhaps saying it's for "safety" as opposed to being blatant Orwellian spying.
deleted 2y ago
[deleted]
richardw 2y ago
Apple might have an advantage given they’ll have custom hardware to drive it, and the ability to combine data from outside the browser with data inside it. But it’s an interesting idea.
- jchw 2y ago
  Apple may have a bit of a lead in getting it actually deployed end-to-end but given the number of times I've heard "AI accelerator" in reference to mobile processors I'm pretty sure that silicon with 'NPUs' are probably all over the place already, and if they're not, they certainly will be, for better or worse. I've got a laptop with a Ryzen 7040, which apparently has XDNA processors in it. I haven't a damn clue how to use them, but there is apparently a driver for it in Linux[1]. It's hard to think of a mobile chipset launch from any vendor that hasn't talked about AI performance in some regards, even the Rockchip ARM processors seem to have "AI engines". This is one of those places where Apple's vertical integration has a clear benefit, but even as a bit of a skeptic regarding "AI" technology, it does seem there's a good chance that accelerated ML inference is going to be one of the next battlegrounds for processor mobile performance and capability, if it hasn't started already. [1]: https://github.com/amd/xdna-driver https://github.com/amd/xdna-driver
  - richardw 2y ago
    For sure many devices will have them, but the trick will be to build this local web model in a way that leverages all of the local chips. Apple’s advantage is in not having to worry about all that. It has a simpler problem and better access to real local data. Give my personal local data to a model running in the browser? Just feels a bit more risky.
    - rfoo 2y ago
      > in a way that leverages all of the local chips Which, in a way, is similar to building a browser leveraging all of the local GPUs to do render and HW-accelerated video decoding. Is Safari on Apple Silicon better than Chrome on random Windows laptop for playing YouTube in the last 5 years? Hardly.
      - cuu508 2y ago
        The video plays fine on both, yes, but the Windows laptop, generally speaking, gets hotter, and runs out of battery sooner.
beefnugs 2y ago
What do you think the bullshit word will be instead of "incognito" this time?

nektro 2y ago

> LLM-powered apps are here to stay, i dont like that you take this as a given. we as users choose. > I don't think this is a terrible idea. it's deplorable

smolder 2y ago

It definitely is a terrible idea, but it follows naturally from the "browser is an operating system" approach the industry has been taking for quite a while. With this, Goog gets to offload AI stuff to clients, but can (and will, I guarantee) sample the interactions, calling it "telemetry" and perhaps saying it's for "safety" as opposed to being blatant Orwellian spying.

deleted 2y ago

[deleted]

richardw 2y ago

Apple might have an advantage given they’ll have custom hardware to drive it, and the ability to combine data from outside the browser with data inside it. But it’s an interesting idea.

jchw 2y ago
Apple may have a bit of a lead in getting it actually deployed end-to-end but given the number of times I've heard "AI accelerator" in reference to mobile processors I'm pretty sure that silicon with 'NPUs' are probably all over the place already, and if they're not, they certainly will be, for better or worse. I've got a laptop with a Ryzen 7040, which apparently has XDNA processors in it. I haven't a damn clue how to use them, but there is apparently a driver for it in Linux[1]. It's hard to think of a mobile chipset launch from any vendor that hasn't talked about AI performance in some regards, even the Rockchip ARM processors seem to have "AI engines". This is one of those places where Apple's vertical integration has a clear benefit, but even as a bit of a skeptic regarding "AI" technology, it does seem there's a good chance that accelerated ML inference is going to be one of the next battlegrounds for processor mobile performance and capability, if it hasn't started already. [1]: https://github.com/amd/xdna-driver https://github.com/amd/xdna-driver
- richardw 2y ago
  For sure many devices will have them, but the trick will be to build this local web model in a way that leverages all of the local chips. Apple’s advantage is in not having to worry about all that. It has a simpler problem and better access to real local data. Give my personal local data to a model running in the browser? Just feels a bit more risky.
  - rfoo 2y ago
    > in a way that leverages all of the local chips Which, in a way, is similar to building a browser leveraging all of the local GPUs to do render and HW-accelerated video decoding. Is Safari on Apple Silicon better than Chrome on random Windows laptop for playing YouTube in the last 5 years? Hardly.
    - cuu508 2y ago
      The video plays fine on both, yes, but the Windows laptop, generally speaking, gets hotter, and runs out of battery sooner.

beefnugs 2y ago

What do you think the bullshit word will be instead of "incognito" this time?