3 ms·

Will this make it easier to ship ML models to consumers? Let's say I'm making a photo editor, can I ship trained models for various image effects and generation

Will this make it easier to ship ML models to consumers? Let's say I'm making a photo editor, can I ship trained models for various image effects and generation using this, and it will run on the client's best available hardware on Windows, Mac OS, Android, etc?

sanxiyn 4y ago
Sadly, I don't think so. Android already has NNAPI, but this post doesn't mention NNAPI at all. This seems focused on running on servers, instead of running on user devices.
- sorenjan 4y ago
  That would be a shame. Where does Apple fit in then, are they running custom ML frameworks in their cloud?
- scotttodd 4y ago
  The post itself doesn't get into too many details about edge deployment, but we're building the core technology to scale down to embedded systems without compromising on feature support. In fact, resource constrained devices really benefit from ahead-of-time compilation to improve binary size and finely control memory usage. We (IREE) published a paper focusing on such embedded device uses: https://arxiv.org/abs/2205.14479 https://arxiv.org/abs/2205.14479.
mikaraento 4y ago
We've shared some plans for client-side inference (TFLite) support in https://www.w3.org/2022/11/17-webmachinelearning-minutes.html https://www.w3.org/2022/11/17-webmachinelearning-minutes.htm.... The presentation was for WebML but mentions non-WebML work. It's not shipping yet though. (I work for Google and I work on client-side StableHLO, but I don't speak for Google).
JacquesPienaar 4y ago
That's a very good question and definitely something of interest. Note, that the compiler is only part of this story (as Mika also mentioned here). With OpenXLA we want to be able to take advantage of the best of what each platform can provide and opsets like StableHLO are meant to provide a portability layer while being expressive enough that targeting specialized hardware efficiently is possible. If you look inside the openxla/iree repo (as well as iree/iree-samples and iree/iree-jax repos, paper Scott cited or sers of IREE like Shark (https://github.com/nod-ai/SHARK#quick-start-for-shark-stable-diffusion-for-windows-1011-users https://github.com/nod-ai/SHARK#quick-start-for-shark-stable...)) you'll see some example.

sanxiyn 4y ago

Sadly, I don't think so. Android already has NNAPI, but this post doesn't mention NNAPI at all. This seems focused on running on servers, instead of running on user devices.

sorenjan 4y ago
That would be a shame. Where does Apple fit in then, are they running custom ML frameworks in their cloud?
scotttodd 4y ago
The post itself doesn't get into too many details about edge deployment, but we're building the core technology to scale down to embedded systems without compromising on feature support. In fact, resource constrained devices really benefit from ahead-of-time compilation to improve binary size and finely control memory usage. We (IREE) published a paper focusing on such embedded device uses: https://arxiv.org/abs/2205.14479 https://arxiv.org/abs/2205.14479.

mikaraento 4y ago

We've shared some plans for client-side inference (TFLite) support in https://www.w3.org/2022/11/17-webmachinelearning-minutes.html https://www.w3.org/2022/11/17-webmachinelearning-minutes.htm.... The presentation was for WebML but mentions non-WebML work. It's not shipping yet though. (I work for Google and I work on client-side StableHLO, but I don't speak for Google).

JacquesPienaar 4y ago

That's a very good question and definitely something of interest. Note, that the compiler is only part of this story (as Mika also mentioned here). With OpenXLA we want to be able to take advantage of the best of what each platform can provide and opsets like StableHLO are meant to provide a portability layer while being expressive enough that targeting specialized hardware efficiently is possible. If you look inside the openxla/iree repo (as well as iree/iree-samples and iree/iree-jax repos, paper Scott cited or sers of IREE like Shark (https://github.com/nod-ai/SHARK#quick-start-for-shark-stable-diffusion-for-windows-1011-users https://github.com/nod-ai/SHARK#quick-start-for-shark-stable...)) you'll see some example.