3 ms·
It's really terrible that Apple markets this as the next big thing but forgets to include detailed documentation so people have to experiment and figure out wha
by bitL 4y ago
It's really terrible that Apple markets this as the next big thing but forgets to include detailed documentation so people have to experiment and figure out what works...
- barkingcat 4y agoApple didn’t “forget” they never want to ever release apple proprietary docs. It’s their competitiveness advantage/ moat.
- smoldesu 4y agoIt's a damn shame, too. If we got the APFS documentation we were promised I feel like we'd be one step closer to world peace.
- lonelyasacloud 4y agoPart Apple's docs haven't been great for a while, part that's just how they roll, and part trying (like most everyone) to figure out what their strategy is going to be in a post GPT4 world [0]. [0] Persist with their own models running locally, how much to integrate with rest of the OS and maintain privacy moral ground, that sort of thing.
- cynicalsecurity 4y agoProprietary software, dude. It really sucks.
- saagarjha 4y agoPeople don't have to do anything. You use CoreML to program it.
- grupthink 4y agoIt's not that simple. If you have a model that actually does something useful (e.g. not just doing matmul & conv2d) your model will fail to run on ANE and, instead, the device will move it over to CPU/GPU and turn your iPhone into a heater. I literally had to continually wipe down my iPhone with a wet towel to keep it from overheating so I could build, ct.convert, run, and debug a model I was working on. Apple doesn't document how to keep operations on ANE. A model created by coremltools may run on either CPU, GPU, ANE, but you don't get to choose. And, if you don't know what you're doing and naively build a model, you will likely run on CPU/GPU only. If your batch is too large, too small, if you need to transpose tensors, if you need to expand mismatched tensors to matmul them together, if your model has an IF branch, or a loop, if you breathe the wrong way, your model silently falls off ANE. But you don't know what caused it. You have to open Netron and guess. Also, it may run on ANE on one device, but not another. There's no documentation from Apple about any of this. So, no, you do not simply "use CoreML to program it".
- saagarjha 4y agoFair enough :) I guess this is difficult to solve for APIs that try to automatically run things on heterogeneous hardware.
- nhubbard 4y agoIt's probably more nuanced than what everyone else is saying. The Neural Engine compute block can change pretty significantly from one chip generation to the next, and instead of exposing the unstable raw capabilities, they use CoreML as an abstraction to keep the changes out of sight. Is it rather annoying? Yes, but it's a more stable method to keep everyone using the Neural Engine from having their software break regularly.