5 ms·

This is a bizarre result but so.. what's the conclusion? That only a few things like Apple's proprietary image lookup are able to tap into the ANE so far? Or th

This is a bizarre result but so.. what's the conclusion? That only a few things like Apple's proprietary image lookup are able to tap into the ANE so far? Or that it's actually just a marketing gimmick? Reading this makes me wonder if it's not just a placeholder for some kind of intrusive system that will neural-hash everything you own, but I'm sure I'm just being paranoid.

londons_explore 5y ago
> That only a few things like Apple's proprietary image lookup are able to tap into the ANE so far? That would seem like the logical conclusion. Perhaps there are hardware bugs/shortcomings that makes it very hard to use for the neural network API. Perhaps the software team is just behind and still building that.
- my123 5y ago
  Using CoreML has some catches, like having to use FP16 instead of integer formats notably.
- noduerme 5y ago
  My initial understanding was that Adobe was leveraging it in their experimental "AI" Photoshop plugins. Although a majority of those seem to require a live internet connection to work. Some of the newer core Photoshop functionality for intelligent selection is ridiculously fast on an M1 Max, though, which makes me think it's probably using the neural chips.
  - sharikous 5y ago
    It might be, but there is also another matrix multiplication accelerator that could be responsible
my123 5y ago
The ANE is accessible via the CoreML framework, it's a high level interface for ML inference. It however turns out that a _lot_ of customer apps today don't use those accelerators at all. (and about the attempt at using BNNS functions, that's not offloaded, it runs on the host CPU cores w/ the AMX tightly bound accelerator)
- galangalalgol 5y ago
  Would it be possible for someone ro write a onnx runtime utilizing coreml? That would open it up to a lot more applications instantly.
  - ianai 5y ago
    ONNX lists CoreML support.
sharikous 5y ago
I have the same anxiety but I don't think it is still there. The Asahi project is putting a deliberate low priority on the ANE but I have seen some other small reverse engineering attempts. I think some use of the ANE outside of Apple APIs will be possible soon.
bayindirh 5y ago
Tensorflow has a CoreML enabled version which run on ANE. https://github.com/apple/tensorflow_macos https://github.com/apple/tensorflow_macos
- my123 5y ago
  AFAIK that doesn't run on ANE but on the GPU. ANE is used for inference only w/ CoreML.
avianlyric 5y ago
I would guess that the ANE has some very specialised hardware (E.g. INT8 or FP16 only), and there’s isn’t a huge amount of it. So many nets either don’t fit completely, or aren’t using the right types of operation to match the ANE. Either can’t run on the ANE, or only have a subset of layers that can run on the ANE. So when running a neural net iOS / macOS needs to make a decision about where to run each net. Even if a net has layers in it that are a perfect match for the ANE, there’s still a trade off from having move the workload back and forth between a CPU core and the ANE (although the unified memory should eliminate a big chunk of this cost). It might be that in the general case it’s not worth the latency hit from using mixed processors when running net that isn’t 100% ANE compatible, or could just be that Apple haven’t got round to implementing the logic needed to gracefully spilt workloads across the ANE and a CPU core. Which would make sense, because they’ve got the time and expertise to ensure all their nets fit within the ANE. Something that’s difficult for 3rd party devs to do, because they don’t have access to detailed ANE docs.
masklinn 5y ago
> Reading this makes me wonder if it's not just a placeholder for some kind of intrusive system that will neural-hash everything you own, but I'm sure I'm just being paranoid. It’s also actively counter-productive: if they wanted to do this sort of tracking, they could just have done what all of their competitors do and send the data straight to their servers. This is hardware (and thus expenses) which is only necessary because of their stance on privacy, and avoiding off-device work.
- mikotodomo 5y ago
  Apple has made huge compromises to try and give some privacy back to the consumer, who lost it all from paying for cheap products where the product is you. And people just ignore this progress because they want to be anti-Apple. It's sad.
deleted 5y ago
[deleted]

londons_explore 5y ago

> That only a few things like Apple's proprietary image lookup are able to tap into the ANE so far? That would seem like the logical conclusion. Perhaps there are hardware bugs/shortcomings that makes it very hard to use for the neural network API. Perhaps the software team is just behind and still building that.

my123 5y ago
Using CoreML has some catches, like having to use FP16 instead of integer formats notably.
noduerme 5y ago
My initial understanding was that Adobe was leveraging it in their experimental "AI" Photoshop plugins. Although a majority of those seem to require a live internet connection to work. Some of the newer core Photoshop functionality for intelligent selection is ridiculously fast on an M1 Max, though, which makes me think it's probably using the neural chips.
- sharikous 5y ago
  It might be, but there is also another matrix multiplication accelerator that could be responsible

my123 5y ago

The ANE is accessible via the CoreML framework, it's a high level interface for ML inference. It however turns out that a _lot_ of customer apps today don't use those accelerators at all. (and about the attempt at using BNNS functions, that's not offloaded, it runs on the host CPU cores w/ the AMX tightly bound accelerator)

galangalalgol 5y ago
Would it be possible for someone ro write a onnx runtime utilizing coreml? That would open it up to a lot more applications instantly.
- ianai 5y ago
  ONNX lists CoreML support.

sharikous 5y ago

I have the same anxiety but I don't think it is still there. The Asahi project is putting a deliberate low priority on the ANE but I have seen some other small reverse engineering attempts. I think some use of the ANE outside of Apple APIs will be possible soon.

bayindirh 5y ago

Tensorflow has a CoreML enabled version which run on ANE. https://github.com/apple/tensorflow_macos https://github.com/apple/tensorflow_macos

my123 5y ago
AFAIK that doesn't run on ANE but on the GPU. ANE is used for inference only w/ CoreML.

avianlyric 5y ago

I would guess that the ANE has some very specialised hardware (E.g. INT8 or FP16 only), and there’s isn’t a huge amount of it. So many nets either don’t fit completely, or aren’t using the right types of operation to match the ANE. Either can’t run on the ANE, or only have a subset of layers that can run on the ANE. So when running a neural net iOS / macOS needs to make a decision about where to run each net. Even if a net has layers in it that are a perfect match for the ANE, there’s still a trade off from having move the workload back and forth between a CPU core and the ANE (although the unified memory should eliminate a big chunk of this cost). It might be that in the general case it’s not worth the latency hit from using mixed processors when running net that isn’t 100% ANE compatible, or could just be that Apple haven’t got round to implementing the logic needed to gracefully spilt workloads across the ANE and a CPU core. Which would make sense, because they’ve got the time and expertise to ensure all their nets fit within the ANE. Something that’s difficult for 3rd party devs to do, because they don’t have access to detailed ANE docs.

masklinn 5y ago

> Reading this makes me wonder if it's not just a placeholder for some kind of intrusive system that will neural-hash everything you own, but I'm sure I'm just being paranoid. It’s also actively counter-productive: if they wanted to do this sort of tracking, they could just have done what all of their competitors do and send the data straight to their servers. This is hardware (and thus expenses) which is only necessary because of their stance on privacy, and avoiding off-device work.

mikotodomo 5y ago
Apple has made huge compromises to try and give some privacy back to the consumer, who lost it all from paying for cheap products where the product is you. And people just ignore this progress because they want to be anti-Apple. It's sad.

deleted 5y ago

[deleted]