5 ms·
Apple mentions TensorFlow explicitly in the ongoing presentation due to the new 16-core "Neural Engine" embedded in the M1 chip. Now that's an angle I did not e
by ojosilva 6y ago
Apple mentions TensorFlow explicitly in the ongoing presentation due to the new 16-core "Neural Engine" embedded in the M1 chip. Now that's an angle I did not expect on this release. Sounds exciting!
Edit: just to clarify, the Neural Engine itself is not really "new":
> The A11 also includes dedicated neural network hardware that Apple calls a "Neural Engine". This neural network hardware can perform up to 600 billion operations per second and is used for Face ID, Animoji and other machine learning tasks.[9] The neural engine allows Apple to implement neural network and machine learning in a more energy-efficient manner than using either the main CPU or the GPU.[14][15] However, third party apps cannot use the Neural Engine, leading to similar neural network performance to older iPhones.
Source: https://en.wikipedia.org/wiki/Apple_A11#Neural_Engine https://en.wikipedia.org/wiki/Apple_A11#Neural_Engine
- wmf 6y agoIt's kind of sad that AMD spent years to get mediocre TensorFlow support and Apple walks in with this. It really shows how huge Apple is.
- WanderPanda 6y agoBut they needed to, MacBooks are simply no option if you want to train models. I dont expect crazy performance but would be great if MacBooks would be an option again for prototyping / model development at least
- stefan_ 6y agoWho mentioned training? Most of these chips are only any good for inference. A wonderful symphony with the Apple computing mantra, of course.
- pantulis 6y agoTotally agree. Besides, can the neural engine be used to speed up other tasks?
- vosper 6y agoCould this also be remedied by Apple supporting Nvidia GPUs again? And then you plug in a beefy eGPU?
- WanderPanda 6y agoI think it is interesting that Apple is at Nvidias mercy again with the Nvidia ARM deal. Hopefully they get their shit together
- danudey 6y agoIn fairness, it's been possible to convert a TensorFlow model to a CoreML model for a while, and in April TensorFlow Lite added a CoreML delegate to run models on the Neural Engine. https://blog.tensorflow.org/2020/04/tensorflow-lite-core-ml-delegate-faster-inference-iphones-ipads.html https://blog.tensorflow.org/2020/04/tensorflow-lite-core-ml-... So don't think of it as Apple walked right in with this so much as Apple has been shipping the neural engine for years and now they're finally making it available on macOS.
- jonplackett 6y agoI can't see how something that tiny can compete in any meaningful way with a giant nVidia type card for training. I'd imagine it's more for running models that have been trained already, like all the stuff they mentioned with Final Cut.
- skohan 6y agoYeah I would imagine it's intended for similar use-cases as they use for iOS - for instance image/voice/video processing using ML models, and maybe for playing around with training, but it's not going to compete with a discreet GPU for heavy-duty training tasks
- ojosilva 6y agoFor an 18-hour battery life computer (Macbook Air) that now doesn't even have a fan, it's for a complete different market segment from where nvidia cards dwell.
- SoSoRoCoCo 6y agoNot all NN models are behometh BERTs, U-Nets or ResNets. Person detection, keyword spotting, anomaly detection... there are lots of smaller neural nets that can be accelerated by a wide range of hardware.
- mmm_grayons 6y agoCan anyone who knows about machine learning hardware comment on how much faster dedicated hardware is as opposed to, say, a vulkan compute shader?
- qayxc 6y agoThat depends entirely on the hardware of both the ML accelerator and the GPU in question, as well as model architecture, -data and -size. Unfortunately Apple was very vague when they described the method that yielded the claimed "9x faster ML" performance. They compared the results using an "Action Classification Model" (size? data types? dataset- and batch size?) between an 8-core i7 and their M1 SoC. It isn't clear whether they're referring to training or inference and if it took place on the CPU or the SoC's iGPU and no GPU was mentioned anywhere either. So until an independent 3rd party review is available, your question cannot be answered. 9x with dedicated hardware over a thermally- and power constrained CPU is no surprise, though. Even the notoriously weak previous generation Intel SoCs could deliver up to 7.73x improvement when using the iGPU [1] with certain models. As you can see in the source, some models don't even benefit from GPU acceleration (at least as far as Intel's previous gen SoCs are concerned). In the end, Apple's hardware isn't magic (even if they will say otherwise;) and more power will translate into higher performance so their SoC will be inferior to high-power GPUs running compute shaders. [1] https://software.intel.com/content/www/us/en/develop/articles/accelerate-deep-learning-inference-with-integrated-intel-processor-graphics-rev-2-0.html https://software.intel.com/content/www/us/en/develop/article...
- deleted 6y ago[deleted]
- akhilcacharya 6y agoYeah, I was confused at that implication. I don't think these are designed for training! (If they are or can be, I'm interested)
- navanchauhan 6y ago> (If they are or can be, I’m interested) Exactly. Currently I am training my models using Google Colab and then exporting the model to run on my MBP. Would be interesting if I could do it locally Another interesting thing is that ( if this is for training ) this will become the only accelerated version of Tensorflow for macOS as: - No CUDA drivers for latest macOS - AMD ROCHm only supports Linux runtime
- mch82 6y agoI’m hoping the M1 can be used for prototyping with small data sets, then final training on Google Colab with complete data sets.
- therealmarv 6y agoIs this just opinion? Maybe they are designed ALSO for training. I wonder if this things can replace nVidia graphic cards on training? The neural core has a LARGE area on the chip design similar to the GPU area.
- cma 6y agoI think this is for inference not learning, even though they use the term machine learning. They seem to just mean running models based on machine learning approaches. Tensorflow includes stuff for inference.
- dionisio 6y agoMy thinking was along the same line as yours, but the way apple framed it seems to suggest that the M1 accelerates model training and not just inference. Here's the actual quote "It also makes Mac Mini a great machine for developers, scientist and engineers. Utilizing deep learning technologies like tensorflow ... which are now accelerated by M1". It should be pretty straightforward to test this though: installing tensorflow-gpu on a mac mini and seeing the result. I suspect, TF's latest branch should also indicate which GPUs are supported. Curious to hear more thoughts on this.
- deleted 6y ago[deleted]
- villgax 6y agoThey do not have any hardware combination which can actually support even modest GPU intensive training sadly, so much touting running models instead of training.
- suyash 6y agoYes, need to see more strong evidence that the new MBP's can handle large amounts of ML Training using TF or CreateML so we don't have to get NVIDIA machines/laptops.
- adammenges 6y agoWould love to get GPU accelerated learning in Lobe (http://www.lobe.ai http://www.lobe.ai).