4 ms·

The second sentence contradicts your statement This soon will be available to all Gemini Advanced subscribers on Android devices, and today we’re bringing it t

by ipsum2 2y ago

The second sentence contradicts your statement This soon will be available to all Gemini Advanced subscribers on Android devices, and today we’re bringing it to more people, starting with all Gemini app users on Pixel 9 and Samsung Galaxy S25 devices

bytesandbits 2y ago
curious, what is done on-device?
- echelon 2y ago
  Likely nothing. Mobile devices can't do much AI stuff except for the tiniest of models, and it'll likely be a long time before they will be able to anything super useful. Edge compute is a long ways off, even on desktop without a dedicated GPU. But especially on mobile. You might have bumped into a media website trying to run a WASM-powered onnx runtime background removal tool, or perhaps a super slim LLM. You'll notice how slow these are and how they can lock up your browser. That's about the experience you can expect from edge compute. Nvidia's proclamation that they're going to be working on robotics as their next growth sector could mean more innovation on the edge / low power compute front. But most of the yield will come from better model architectures and models designed specifically to work with compute constraints. For now, datacenter inference reigns supreme.
  - pzo 2y ago
    > WASM-powered onnx runtime background removal tool Yes it's slow because they will most likely execute on CPU (for sure on iOS). WebGPU is still not enabled on safari and WebNN is not even supported anywhere (so that you can use NPU provider). ONNXRuntime is not even the most optimised when running natively (instead of WASM) on iOS, e.g. doesn't support MPS provider (GPU) and NPU provider (via CoreML) implements only subset operators (last time I tried). Safari also provide limitation on WASM memory usage In practice when you want the best performance you would have to use native app and CoreML with NPU provider and model architecture optimized for NPU. On iOS for now big limitation is available RAM, but even my iPhone 13 mini has exactly the same fast NPU as on my Macbook M2 Max when tested it's having similar speed to running on GPU.
- deleted 2y ago
  [deleted]
addandsubtract 2y ago
I still can't use the "summarize" feature that was used in marketing to sell the Pixel 8 Pro. Not holding my breath on ever being able to use this, either.

by ipsum2 2y ago

bytesandbits 2y ago

curious, what is done on-device?

echelon 2y ago
Likely nothing. Mobile devices can't do much AI stuff except for the tiniest of models, and it'll likely be a long time before they will be able to anything super useful. Edge compute is a long ways off, even on desktop without a dedicated GPU. But especially on mobile. You might have bumped into a media website trying to run a WASM-powered onnx runtime background removal tool, or perhaps a super slim LLM. You'll notice how slow these are and how they can lock up your browser. That's about the experience you can expect from edge compute. Nvidia's proclamation that they're going to be working on robotics as their next growth sector could mean more innovation on the edge / low power compute front. But most of the yield will come from better model architectures and models designed specifically to work with compute constraints. For now, datacenter inference reigns supreme.
- pzo 2y ago
  > WASM-powered onnx runtime background removal tool Yes it's slow because they will most likely execute on CPU (for sure on iOS). WebGPU is still not enabled on safari and WebNN is not even supported anywhere (so that you can use NPU provider). ONNXRuntime is not even the most optimised when running natively (instead of WASM) on iOS, e.g. doesn't support MPS provider (GPU) and NPU provider (via CoreML) implements only subset operators (last time I tried). Safari also provide limitation on WASM memory usage In practice when you want the best performance you would have to use native app and CoreML with NPU provider and model architecture optimized for NPU. On iOS for now big limitation is available RAM, but even my iPhone 13 mini has exactly the same fast NPU as on my Macbook M2 Max when tested it's having similar speed to running on GPU.
deleted 2y ago
[deleted]

addandsubtract 2y ago

I still can't use the "summarize" feature that was used in marketing to sell the Pixel 8 Pro. Not holding my breath on ever being able to use this, either.