12 ms·
Apple M1 support for TensorFlow 2.5 pluggable device API
- 542458 5y agoVery cool. I’m not an ML guy, so apologies if this is a dumb question: how good is performance compared to other hardware here?
- rubatuga 5y agoNot a lot of benchmarks, but I heard large models struggle on M1.
- judge2020 5y agoHave a reference perhaps? I imagine it would be severely limited by memory size & speed for large datasets, but there might be other factors as well.
- teruakohatu 5y agoIt is much, much slower than a dedicated GPU or TPU, so it is useful for "edge" inference eg. local speech or image recognition, but not for training new models. It is using shared memory with the CPU so I don't know how much memory you could eat up while training. In short, you will probably get much better performance using Google Colab to train a deep learning model, than the M1.
- goodcjw2 5y agoThis is pretty cool to have more generic GPU-based support. Might also be related: Tensorflow Lite Core ML delegate enables running TFLite model on CoreML enabled iOS devices. https://www.tensorflow.org/lite/performance/coreml_delegate https://www.tensorflow.org/lite/performance/coreml_delegate Would nice to see performance comparisons on M1 Mac/iPad for which way is more performance and efficient. (Admittedly TF vs TF-lite is a 100% apple to apple comparison, pun-intended).
- b9a2cab5 5y agoJust running native models on OpenCL has existed for a while now, through PlaidML and other companies. Unfortunately you need CUDA support for custom kernels for a lot of modern architectures like Transformers, and AMD's CUDA transpiler leaves a lot to be desired.
- codelord 5y agoM1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance is 35.5 TFLOPS.
- jra101 5y agoVega 20 should be ~13.8 TFLOPs (single precision): https://www.anandtech.com/show/13923/the-amd-radeon-vii-review https://www.anandtech.com/show/13923/the-amd-radeon-vii-revi...
- codelord 5y agoSeems like AMD has been using Vega 20 to refer to two different things. I was talking about the mobile GPU on MacBook Pros which is based on a 14nm chip. The full name is Radeon Pro Vega 20: https://www.amd.com/en/graphics/radeon-pro-vega-20-pro-vega-16 https://www.amd.com/en/graphics/radeon-pro-vega-20-pro-vega-... https://www.techpowerup.com/gpu-specs/radeon-pro-vega-20.c3263 https://www.techpowerup.com/gpu-specs/radeon-pro-vega-20.c32... Vega 20 seems to also refer to a discrete GPU. This has been later rebranded to Radeon VII (maybe because of this confusion). The number you are quoting is for the discrete GPU.
- jra101 5y agoHuh, I had no idea they used Vega 20 both as a codename and a product name. Confusing.
- floatboth 5y agoSame with 10 LOL. VEGA10 codename is for the original Vega Frontier Edition/RX Vega 56/RX Vega 64. But now there's also "Vega 10" used as a description for the 10 Vega compute units on Ryzen APUs.
- rozim 5y agoI have found the M1 air fine for web browsing but kind of hard to install software on. Following the instructions: ----- python -m pip install tensorflow-macos ... ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly ----- (base) dave@daves-air ~ % uname -a Darwin daves-air.lan 20.5.0 Darwin Kernel Version 20.5.0: Sat May 8 05:10:31 PDT 2021; root:xnu-7195.121.3~9/RELEASE_ARM64_T8101 arm64
- qmmmur 5y agoI run everything python under Rosetta. Easiest way is to install home brew for Intel processors then make an alias to that version for managing rosetta stuff
- easton 5y agoYou’re missing out on a gigantic speed boost. If you haven’t checked lately check again, most of the big libraries have been recompiled over the last few months (tensorflow excluded apparently).
- fastball 5y agoKinda silly to use Rosetta when the whole point of this is native support for TensorFlow on M1.
- xnyan 5y agoMaybe, but lots and lots of software applications are not even close to being hardware constrained or in this specific case Rosetta constrained. If you're trying to get work done and you have the computational overhead available, this absolutely sounds like the right solution. Even if it's less efficient, I can easily imagine situations where "I need this to work without thinking about it" could win out.
- qmmmur 5y agoYeah I am not running code which is hardware constrained and numpy was a real doozy tto get working because of pep517. I found Rosetta pretty fast anyway. Certainly faster than my Ryzen 2600 on arch Linux at running the same code.
- dandiep 5y agoI was able to install this fairly easily (much more so then the crap they dumped out here - https://github.com/apple/tensorflow_macos https://github.com/apple/tensorflow_macos. Just take a look at the 200 github issues that were ignored for the most part...) I also noticed that in my project I got a decent speedup immediately when executing my model, but I have not run any benchmarks. But, where do you go to file bugs? Ask questions? etc. I am not a big Mac developer, so is there something I don't know?
- eschaton 5y agoYou can file bugs using the Feedback Assistant app in /System/Library/CoreServices/Applications, and you can ask questions on the Developer Forums at <https://developer.apple.com/forums/ https://developer.apple.com/forums/>. Both will require you to have a (free) Apple ID.
- NmAmDa 5y agoI think they don't track bugs there. They even archived the repository.
- sillysaurusx 5y agoAnyone know if this shows up as an actual GPU device? The last tensorflow-macOS thing did not. If you list devices using that, you’ll see only one: CPU:0. Does this give a GPU:0 device? You can check via: import tensorflow as tf2 from pprint import pprint as pp tf = tf2.compat.v1 sess = tf.InteractiveSession() pp(sess.list_devices()) I’d check myself, but I’ve been so burnt by tensorflow 2 and M1 problems that I just don’t have the energy to figure out the inevitable compilation issues, and it sounds like at least one other person already has it running. Plus I’m on mobile.
- liprais 5y agoPython 3.9.4 | packaged by conda-forge | (default, May 10 2021, 22:10:52) [Clang 11.1.0 ] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow Init Plugin Init Graph Optimizer Init Kernel >>> tensorflow.config.list_physical_devices <function list_physical_devices at 0x128011f70> >>> tensorflow.config.list_physical_devices() [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
- sillysaurusx 5y agoOhhhh snapppppp! Excitement level just shot way up. Thank you. God, it’ll be nice having a gpu for tensorflow. I’ve dreamed about this for like… a long time.
- sillysaurusx 5y agoGot it running! I posted detailed instructions here: https://news.ycombinator.com/item?id=27445555 https://news.ycombinator.com/item?id=27445555 Ahh, I've never had a local GPU before. Thank you Apple Santa Claus, whoever your team is.
- dekhn 5y ago... and it worked on linux and windows for years already.
- HerbsMan 5y agoOnly Python?
- troyharvey 5y agoI have some notes on doing ML/python work on M1 Macs that may be helpful. https://twitter.com/troyharvey/status/1380586300911280128?s=21 https://twitter.com/troyharvey/status/1380586300911280128?s=...
- tak5651 5y agofinally
- Labo333 5y agoCan't wait to get metal support in PyTorch!
- sydthrowaway 5y agoI find the benchmarks confusing. If we normalize, is Apple close to beating Nvidia?
- mjburgess 5y agoNot on M1. However as M1 is a chiplet design, it's pieces are stackable like legos (to some degree). It is then supposed that with enough stacking, apple could get close to a dedicated GPU performance. However this is speculative.
- kimixa 5y agoM1 isn't a chiplet is it? It's a monolithic SoC, and while it is packaged with the RAM that's just closer integration of chips that would normally be on-motherboard or a DIMM?
- mjburgess 5y agoThe M1 itself is an SoC. It's composed of parts, however, which can stack. Eg., just look at the M1 variations present atm: variable gpu cores. The "chiplet" speculation is that given the M1's variable high-perf, low-perf and cpu cores, this can scale up. Leaks at least are all pointing in a 32-core, Xeon-competitor direction. It is theoretically possible they could do the same with GPU count, and try to compete perhaps in the mid-pro gfx range.
- floatboth 5y agoBasically all silicon works like this. "Variable cores" actually means some cores are disabled. This is usually done to increase overall yields: chips with damage inside one of the cores can still be binned as the lower core count SKUs. The "you can scale up" thing is actually just how e.g. Intel makes bigger monolithic chips (Xeon/HEDT) with the same or very similar cores as the desktop ones. Meanwhile AMD, actually using chiplets, can cheaply do something more like "scale out" in the sense that they put more of the exact same die on a package.
- 5y ago
- qiqitori 5y agohttps://github.com/tensorflow/tensorflow/releases/tag/v2.5.0 https://github.com/tensorflow/tensorflow/releases/tag/v2.5.0 (Linked from Apple's article) Wow, that list of CVEs is 110 lines.
- deleted 5y ago[deleted]
- tsbinz 5y agoAnd it was released almost a month ago with the CVEs saying "these fixes will be backported to previous branches that are still supported" but releases for those branches haven't happened yet, so if you're on an older version and just want to get the security fixes you have a problem.
- rrss 5y agothe majority of these are "an attacker can craft a model that causes problems." Are people actually using tensorflow to run untrusted models?
- ojosilva 5y agoI'm still trying to find a way to monitor the Neural Engine on my Macbook air M1, but the APIs are non-existent, there's barely anything in the docs and no answer from Apple. My models train fast, 3x faster than most i7 computers with GPU, which is excellent for a fanless ultraportable computer but I wish Apple would treat the NE as a 1st class citizen on these machines, with Mac SDK APIs and usage visualization in the Activity Monitor.
- okamiueru 5y ago> 3x faster than most i7 computers with GPU Can you back that statement up with anything, or at least clarify it? You seem to suggesting a non-mac i7 with a separate GPU. Also, just an FYI, "i7" says pretty much nothing. The i7s have existed since 2009. I don't know. The statement is just so vague and ridiculous. The M1 is probably the worst hardware you could have picked in 2020-2021 if computational power was your main concern. For highly parallelizable work tasks, the top end GPU alone has 10x the computation power than the M1, and a top end CPU has around 4x the computation power than the M1. Not to mention a rather limiting 16GB of memory. That the M1 is computationally powerful is a myth started out by exceedingly misleading marketing and reinforced with hard-to-compare benchmarks. To possibly save someone the trouble, the responses to a comment such as this, from experience will be: 1. "power consumption is much much better on M1 than anything else." True, but, then again, your use case must then prioritize power consumption, and not computation power. So which is it? The use case for a compute cluster on a train is rather contrived. 2. "When apple scales up the M1 to more cores, they will magically be able to retain all the benefits possible with a low core count, and scale it up without any problems or compromises, just you wait.". Ok, I'll wait. 3. "It's not fair to compare a laptop with a desktop". Of course it is. The constraints for comparison are already stated: computation power being the main priority. If someone buys hardware to do heavy computations, you can pick and chose depending on your needs. If you need it to be a laptop, or you need it to draw little power, then I'm sure you can factor this in accordingly.
- jonathanstrange 5y agoI don't know about the M1, to be honest, but I used to be an Apple customer and ardent fanboy (so embarrassing!) during the golden PPC age and remember very well Apple's inflated claims about performance, which all turned out to be false the minute they switched to Intel. So I agree with your comment, it's advisable to always take miraculous performance claims with a grain of salt.
- zimpenfish 5y agoInstalled these - which seemed to work bar a few messages about `numpy` not being installable - but trying to use `textgenrnn` ran into a whole bunch of Keras problems (for which the internet's answer is "use tensorflow.keras" expect `textgenrnn` is already doing that...) I yearn for the day when someone makes a nice, simple, "install this and python ML works fine with your GPU" package.
- albertTJames 5y agoInstall conda then use pip ... does not make much sense. Conda is so clunky.
- ribit 5y agoThe previous beta 2.4 branch would also utilize AMX coprocessor when appropriate. Does anybody know whether the plugin is also based on ML Compute?