3 ms·
As far as I understand, Apple's ML Compute framework cannot use the neural cores in eager mode (only in graph mode). So, the neural cores would probably only be
by danieldk 6y ago
As far as I understand, Apple's ML Compute framework cannot use the neural cores in eager mode (only in graph mode). So, the neural cores would probably only be used with TorchScript.
(The TensorFlow implementation has the same limitation, but using graph execution was traditionally more popular in TensorFlow, since it didn't initially have an eager mode.)
- 0x008 6y agoI thought the tensor flow alpha for mac uses gpu hardware acceleration, not neural cores?
- reedf1 6y agoDoesn't pytorch also include a graph option? Or is it only a graph-like interface?
- danieldk 6y agoYes https://pytorch.org/tutorials/beginner/hybrid_frontend/learning_hybrid_frontend_through_example_tutorial.html https://pytorch.org/tutorials/beginner/hybrid_frontend/learn... https://pytorch.org/docs/stable/jit.html https://pytorch.org/docs/stable/jit.html But people normally use PyTorch in eager mode.
- nl 6y agoI don't understand why this should be the case. It might be that no one has written the code to make it work in eager mode, but I'm trying and failing to think a hardware reason this could be the case.
- atty 6y agoFrom NeurIPS (today, actually), they showed that it works both in graph and eager mode in a short 15 minute talk. However, no discussion about any potential difference in performance.
- danieldk 6y agoInteresting, because the README in the repository states that it will only use CPU cores in eager mode: Please note that in eager mode, ML Compute will use the CPU.
- atty 6y agoI think that those can both be correct, and they were just being a little slippery with their language. During the talk they tried pretty hard to focus on saying their compute engine would pick the best hardware to run on... and I suppose if the GPU has terrible performance for eager mode, then the CPU would be the best. However I find that quite disappointing. I hope there’s significant upgrades coming down the pipeline for when they start releasing more powerful Apple silicon based devices.