23 ms·
Are the New M1 Macbooks Any Good for Deep Learning?
- usmannk 6y agoTraining? No, of course not. It's still a laptop GPU/SoC/"Neural Engine"/whatever. Especially when the alternative is free GPU via Google Colab etc, there is no competition.
- andy_ppp 6y agoI was under the impression things start getting doable at 2 dedicated Nvidia GPUs (preferably with 1tb of ram each)?
- pishpash 6y ago1TB? Isn't it more like 12-64GB?
- muro 6y agoIt was clearly an exaggerating joke.
- ganstyles 6y agoThat's not clear at all, if it was a joke. It definitely doesn't come across as a joke, but perhaps just someone who isn't up to date on gpu memory, which is a fairly niche subject.
- bee_rider 6y agoOne would have to be pretty out of the loop to think that GPU memory was in the terabyte range, though.
- gambiting 6y agoEnterprise grade workstations and servers can be configured with 1TB of ram though, so it's not completely inconceivable that you could have a dedicated GPU workstation with 1TB of graphics ram. I mean the Tesla A100 cards have 40GB of VRam each so you'd only need 26 of them to have 1TB of VRAM in a single machine. While I have no idea if such a machine exists, it certainly could exist, 26 GPUs wouldn't fill even half of a server cabinet. Edit: actually Quattro A6000 cards ship with 48GB vram each, so you only need 22 of those to have 1TB total.
- penagwin 6y agoHere you go friends > Radeon Pro SSG (2016) https://www.amd.com/en/products/professional-graphics/radeon-pro-ssg https://www.amd.com/en/products/professional-graphics/radeon...
- monocasa 6y agoThat's a GPU with a terabyte SSD on it, not a terabyte of VRAM.
- adgjlsfhk1 6y agoIt would be really interesting if micron or intel collabed to make a gpu with a ton of 3d xpoint.
- simias 6y agoI'm not super knowledgeable in machine learning but given that system RAM (not VRAM) in the TB range is really not that rare these days for high end workstations I assumed that the poster meant that you'd have 1TB of working RAM per GPU for the learning job (which would be paged in and out of VRAM as necessary). I guess the lesson to be learned here is that if you want to use an implausibly huge amount of RAM to make a point, a TB is not safe anymore. Go for Exabytes instead, that should be unambiguous for a couple of years.
- bee_rider 6y agoHaha, that might just be the right takeaway!
- tluyben2 6y agoI thought it was but because 1TB is not too far out of the park, it would he been more obvious if something like '1 exabyte gpu' was used. Then, at least for a few years to come, it is indeed crazy talk.
- ganstyles 6y ago1 decent GPU is usually fine for reasonable things. But the memory is way off: the gpus I use have 40GB of memory (a cluster of the higher end Nvidia A100s) and they're basically state of the art. Most gpus near sota are in the 16GB range and up.
- disabled 6y agoYeah, I personally like using an Nvidia Jetson AGX Xavier. There are some cool packages out there, that detect your emotions and attentiveness while driving. Aside from the Jetson, I have an 2080 Super on my laptop.
- armatav 6y agoThere are graphs comparing it to Google Colab GPU/CPU in the article.
- m463 6y agoThe hype is getting out of control. They do have an interesting cpu, but the apple target market isn't engineers/scientists/gamers so they traditionally have not done anything interesting with respect to ml and gpu kinds of things. The closest is video. I think things might get interesting with an arm mac pro if they can add a lot of cores. I wonder if they will add PCIe slots so they can collaborate with other hardware manufacturers, or if they will navel-gaze some more and close it off.
- JulianWasTaken 6y agoIt seems odd to phrase things as "hype" and say Apple doesn't target engineers as their target market. The machine literally has a component in it called a neural engine which Apple advertises as: > In fact, with a powerful 8‑core GPU, machine learning accelerators, and the Neural Engine, the entire M1 chip is designed to excel at machine learning. (https://www.apple.com/mac/m1/ https://www.apple.com/mac/m1/) So asking "how well does that really perform" seems like precisely the right question, and to me it'd seem very clear Apple wants a piece of that market.
- R0b0t1 6y agoUsually a mobile device isn't for training just application.
- comex 6y agoThe Neural Engine is intended solely or almost solely for inference, not training. For instance, in a post from last November [1], Apple mentioned their tensorflow_macos fork can use the "the GPU in both M1- and Intel-powered Macs for dramatically faster training performance", but didn't mention the Neural Engine. (Incidentally, I think that is the same fork used for the benchmark here.) [1] https://machinelearning.apple.com/updates/ml-compute-training-on-mac https://machinelearning.apple.com/updates/ml-compute-trainin...
- p1necone 6y agoThat's for running end user applications that rely on already trained neural nets.
- azinman2 6y agoHow does collab make money? Do you have to eventually pay? Or pay to make it faster? Considering the computationally intensive nature of ML, does it make more sense to train on specialized cloud-based processors?
- zmmmmm 6y agoYes, given Google's reputation, relying on anything run by Google that doesn't make (a lot!) of money is a high risk venture. It's one thing to do so when you have options to fall back to, but if you are buying yourself a machine learning laptop as a long term (2 year+) investment on the basis that you don't need GPU hardware because of Colab ...
- iujjkfjdkkdkf 6y agoCollab is awesome. It's not really for any serious training, it's for evaluating and sharing ML models that (may) need a GPU, and learning. But for many situations, it replaces the need to have a laptop with a GPU - it's good enough for getting set up before you engage the real compute to do serious training. If collab was not free, I would consider paying a subscription for it, which is maybe the end game. They have a paid version now that is supposed to be faster and more lenient about disconnecting you, but I recently read a comparison that said it wasnt worth it.
- aljarry 6y agoI guess it's a testing environment for Google Tensors.
- hamilyon2 6y agoGoogle collab doesn't need to make money. It needs to suffocate space so much, that no potential competitor with commercial offering ever has a chance. The money they make here is money they save by not having to buy competitor for nine figures later.
- logicslave 6y agoM1 chip is a classic case of some big old monopoly (intel) that refuses to innovate. Probably all the execs over there will get wildly rich while they loot the company and drive it into the ground.
- pengaru 6y agoFailing != Refusing
- logicslave 6y agoMost likely there is a bunch of middle management that blocks the innovation. Its a refusal.
- eyelidlessness 6y agoI think it's more likely the same mistakes Intel made when approached to supply chips for the iPhone have persisted: focusing more on "raw performance" over "efficiency". When it's been clear for years, and especially clear now, that efficiency is key to raw performance advancing new processor designs.
- vlovich123 6y agoIt’s not clear to me. Are these M1 tests running on the CPU, GPU, or neural chip? If the M1 CPU is running at half the speed of a Colab GPU that seems impressive if the software for it is still immature enough that the neural engine/GPU aren’t optimized around.
- joakleaf 6y agoThe M1 Tensorflow-branch doesn't use Neural Engine (ANE). However, it does use the M1-GPU. The M1 GPU is about 1/3 the speed of a 1080TI card when looking at OpenCL score in Geekbench, but may perform faster than that in some cases, due to the shared memory architecture of the M1. As I understand it, the ANE has very low precision, which makes it unsuited for training in Tensorflow.
- GeekyBear 6y agoComparing an integrated GPU to a discrete GPU seems a bit Apples to Oranges, if you'll pardon the pun.
- eyelidlessness 6y agoIt's highly unlikely Apple is going to support discrete GPUs in any ARM-based Mac without another major change in business direction. So once the full product line is available, it's going to be as Oranges-to-Oranges as you can reasonably expect.
- GeekyBear 6y agoAn M1 Mac sees and mounts a discrete GPU connected over Thunderbolt. It just doesn't have an ARM flavored driver for it. Which seems to align just fine with Apple's ongoing efforts to get rid of all the kernel mode drivers and replace them with user mode drivers over time. I don't think you should read too much into their entry level chip not supporting external GPUs in the first generation.
- choppaface 6y agoAlternate headline: tensorflow mlcompute on M1 macs actually works. That by itself is notable. The performance still makes Colab look like a better option, but it's been hard to even track the viability of deep learning training on Macs and this is a useful milestone to have.
- CapriciousCptl 6y agoIt’s not a viable offline training machine but this thing flies with data munging. I have the 16gb version and pandas data frames that are 40gb or so load from csvs in two minutes with type inference for example. So long as you can use Google Collab it’s an excellent deep leaning laptop where it counts.
- fock 6y agohow do you load 40GB of data into a device having only 16GB RAM (or do you mean 40GB csv which goes down in size when parsed machineusable?)
- deleted 6y ago[deleted]
- iujjkfjdkkdkf 6y agoA 3 layer convnet probably isnt the last word in whether the M1 is better for "deep learning". Something nontrivial where the gpu/processor is maxed out would be a more interesting comparison, even one of the pytorch vision tutorial that uses resnet I think could be a more realistic DL comparison
- currymj 6y agothere are lots of applications that benefit from GPU acceleration, where even a single small-ish GPU is useful. Even outside of deep learning -- DL tensor libraries can be useful just for GPU numerical operations + autograd. Managing moving data back and forth to cloud environments can add a lot of friction. A laptop that has the performance of a low-end discrete GPU, but is the size and temperature of a regular laptop, would be a very nice thing to have. Hoping software support for the M1 continues to improve.
- jefft255 6y agoI think a better comparison would have been some of the smaller discrete GPU wielding laptop. You can get some of these for similar prices to the M1 macs. Also, for me, the answer is no: they're not any good. I use PyTorch. I use custom CUDA code. The reality is brutal but simple: if it doesn't run CUDA, no serious ML research will use it for training anything else than toy models.
- hu3 6y agoIs there a roadmap for CUDA support? (downvoted for asking a honest question. Ahhh the things you see on Apple related posts... Emotion driven bunch)
- coayer 6y agoCUDA is proprietary and exclusive to Nvidia, so probably not! (Apple and Nvidia haven’t been cooperative for some time)
- whywhywhywhy 6y agoIt's even more absurd because Apple discontinued OpenCL and replaced it with their own proprietary solution too.
- jefft255 6y agoOfficially I don't see that coming, I think it's iffy in terms of patents etc...
- cnasc 6y agoCUDA is Nvidia exclusive, no? I don’t think support for third party graphics hardware will be forthcoming
- russli1993 6y agocuda is designed by nvidia, to lock software and applications to use Nvidia's hardware. Cuda is the core of Nvidia's business model. Cuda is the reason why Nvidia has the guts to charge $5000 for a data center or Quadro card when that card uses nearly the same silicon as the consumer versions and customers will still have to buy their cards. Its like Apple sell you the iPhone, but also the iOS APIs and application model so you can run iOS apps. Once you run iOS apps, you are in Apple's ecosystem, both the ISVs and the user are hard to leave. Its like saying when will Apple officially make iOS APIs and libraries run on Android phones. Both will never happen.
- jshen 6y agoHas anyone done, or seen, any analysis on energy used for ML training rather than clock time? I’ve done some of my own for comp heavy workloads, but it wasn’t ML on a GPU.
- rvz 6y agoMaybe not the M1 but perhaps the M3 or M4 Macs. Who knows. These Macbooks will be superseded anyway, so I'd rather wait until the software I'm using is fully supported and optimised than to jump into the first generation of M1 Macs with unoptimised / unsupported software running in rosetta. But right now, in general? No. they are not good for deep learning.
- chongli 6y ago.
- BatFastard 6y ago> In general, they’re fantastic. The question was "Are they good for deep learning" and the answer was no. No one asked about them in general.
- rvz 6y agoWhy are you running away by hiding your comment? Someone else saved your comment anyway. Next time delete your post if you want to run. > In general, they’re fantastic. The unoptimised software and the missing developer tools says otherwise. Especially for users of deep learning, if it lacks the tools they will not use it at all for this use case. > The question was "Are they good for deep learning" and the answer was no. No one asked about them in general. Don't you think the answer is to skip the M1 altogether and in general for developers and deep-learning users? At this point, there is no reason on getting an M1 Mac at all since the software required is not even ready for M1 and the hardware will almost certainly be obsolete this year for M2. I wouldn't want to be an early adopter on a system that has unoptimised software on it and would be running on Rosetta. The answer to the question above lies in whether if the hardware in the newer generation Mac products is powerful enough for deep-learning. In this case, it is not. So just get a desktop with a RTX 3080 instead or wait for an M4 / M5 Mac.
- futuretaint 6y agoIn general M1 is fastastic tho.
- joshuaengler 6y agoI'd wait a few generations to be honest, Mac laptops have become notoriously faulty in recent years. They're not the bulletproof machines they once were, and tend to have a lot of engineering flaws (flex cable and display issues especially). The M1 seems okay so far as it's an iteration of the previous gen, but they did make a lot of changes as well. I'd hold out for a later version.
- floo 6y agoAppreciate seeing some actual benchmarks. But it is a bit silly to round the results to seconds when they are in the single digit range.
- olliej 6y agoAre any of these benchmarks actually using the M! "ML"/"AI" hardware?
- m15i 6y agoAny thoughts on the value proposition if/when >= 64 GB RAM will be available on apple silicon? Even if the chips are relatively slow, the extra RAM could be worth it.
- personjerry 6y agoIs anyone really doing serious deep learning locally on a laptop these days? I think being able to play with some models is great to be able to do locally, and then you'd probably deploy to your remote setup for the production size training / deployment. I don't feel like this wasn't much of a loss.
- _coveredInBees 6y agoNot everything in deep learning involves running gigantic models on large GPUs or TPUs in the cloud though I'm sure that is the norm for a lot of companies. There is a lot of value in being able to run smaller models when you are working on real-time, edge applications, especially on the inference side of things. We do a lot of that where I work and I find my macbook pro very frustrating on that front after previously having had a Dell XPS 15 with a decent enough nvidia GPU.
- kayoone 6y agoyou can still easily do that on a smaller remote server with a big GPU, there is not really much time overheard in doing that when setup correctly.
- _coveredInBees 6y agoYes, I do that now, but there are still issues around that workflow and nothing beats being able to prototype locally. Some issues (not major deal breakers, but still a source of annoyance): - Much more limited IDE experience if you use any graphical IDE. I prefer Pycharm because it is far superior to pretty much anything else out there when working in Python + Pytorch. You need to use a remote desktop solution like VNC in that situation, which is not remotely close to being on-par with local code prototyping. Jupyter notebooks I abhor because of how poor they are in relation to any decent IDE. VsCode is the only tool that has a great remote dev workflow but it just isn't near the functionality of Pycharm when you use the latter daily. You also don't directly get the ability to plot and visualize data in something like matplotlib when working remotely which is again an issue. I'll still use it if I have to when VNC isn't snappy enough - If data security is a concern, everything needs to occur through a VPN which is another intermediate step in your workflow to get started every time you open up your laptop. - If you have spotty internet access or are traveling, the remote dev workflow suffers immensely. The other thing I should mention is that outside of the standard data exploration + model training/inference workflow, there are other use-cases where being able to prototype locally is very advantageous. I write a lot of our tooling and internal libraries (for example: a keras-like model training framework in Pytorch) and that involves a lot of pytorch code that references GPUs but testing and prototyping that could easily be done with small models on a laptop. Not being able to do that at all is really annoying and having a native IDE experience when writing a library (especially when you rely on several internally developed libraries) is very critical in my experience. Again, all of these are not individual deal-breakers, but I feel the pain almost every day.
- ngcc_hk 6y agoFlash is proprietary to ... it is retiring. I trust if one can one may move on. The community can move on. Not to apple but from nvidia. Obviously hope it is not from nvidia to apple, which is unlikely. One likely is to 2camp so to have competition.
- king_magic 6y agoSure, as a badass SSH machine, it works great for deep learning... when SSH'd to a server with NVIDIA cards. Now don't get me wrong, I absolutely adore my M1 MacBook Air. It's so good that it I've kept my i7+2080+32GB RAM desktop turned off for weeks now (outside of gaming). My 8GB M1 MacBook Air is now my main work computer, and it's spectacular. That said, the basic MNIST timing benchmarks are decent, but don't get your hopes up for anything more anytime soon. There's simply no way in hell today's M1 SoC can train much, much bigger & complex models than hardcore systems with beefy GPUs with 12+ GB RAM on each card. So no, today, the M1 MacBook Airs/Pros are not really good for DL. And honestly, that's fine. Maybe Apple will compete with NVIDIA down the road, and I'd be happy to see that. But I'm glad the author came to the (correct) conclusion that M1s just aren't anywhere near there yet for anything more than basic DL. Edit: that all said, there might be an interesting use case for M1 Mac Minis as edge inferencing nodes.
- debbiedowner 6y agoGoogle colab TPU left out why? Its free just like the other two...
- sandGorgon 6y agoTensorflow is natively accelerated on M1. The branch is different - https://github.com/apple/tensorflow_macos https://github.com/apple/tensorflow_macos Not sure what the confusion was.
- visarga 6y agoCommoditizing their complement - making it easier to teach ML with a hands-on approach, train and demo neural nets, so people and companies end up running them on their cloud machines.
- desmap 6y agoThe title is promising but the comparison is more than subpar and misleading: - There shouldn't be any comparison to a bare CPU because M1 includes a TPU, a typical CPU doesn't - Comparing it to Colab's GPU is good because latter includes a TPU but OP should have stated which GPU he got; Google Colab allocates different GPU models
- KingOfCoders 6y agoAll of this is interesting. I'm currently using - as many do - a 2080TI for ML training. With this my training is <30min. I would not use a laptop for training as I no longer use "developer laptops" that are more expensive with lower performance than my Linux desktop, especially now with homeoffice I no longer need a laptop - I understand others have a need for laptops YMMV. But the benchmarks could give a glimpse for a desktop M1. What I want to know, is it 0.5x or 2x the speed of a 2080TI.
- singularity2001 6y agoJust tried out apples native tensorflow for M1 branch yesterday: https://github.com/apple/tensorflow_macos https://github.com/apple/tensorflow_macos installation was a non-issue, The training kept the Mac mini completely silent, having 40% of colab gpu speed is very satisfying for small tests. pytorch on the other hand is not ready yet (there are installation instructions but they didn't work)
- 0x008 6y agoIs a bicycle great for transporting goods? Yes, absolutely, if your goods are like a handful of oranges. No, absolutely not, if your goods are like super heavy car parts.
- m3at 6y agoInteresting decision to go with tensorflow on a new system. If you have no choice because of existing code it's understandable, though recent research has clearly shifted to pytorch [1]. Though if you're required to use tensorflow, will you even be able to switch to the bleeding edge version? N=1 but all projects I worked on where customers required tensorflow had to be on version <2 http://horace.io/pytorch-vs-tensorflow/ http://horace.io/pytorch-vs-tensorflow/
- DrBazza 6y ago> but these still aren’t machines made for deep learning. Betteridge's law of headlines again.
- viktorcode 6y agoTo be fair, I found another earlier ML preview to be more balanced (please ignore the title): https://towardsdatascience.com/apples-new-m1-chip-is-a-machine-learning-beast-70ca8bfa6203 https://towardsdatascience.com/apples-new-m1-chip-is-a-machi... As evident from the benchmarks, the result is very dependant on your network. Some getting huge acceleration boost from the neural processor, while others can't be accelerated. I would suggest try and see approach.
- jC6fhrfHRLM9b3 6y agoStop supporting Slave labour.
- jzer0cool 6y agoI've been out of the GPU market for a while. Any comparison with M, other laptops and desktop with various GPU card price points? What are some current solutions when doing local development without breaking the wallet too much?
- wapxmas 6y agoApple's hardware is not any good for ML since they abandoned CUDA ecosystem giving nothing to match the needs of such customers.
- AtlasBarfed 6y agoCUDA is owned by NVidia, and they are closed source in all practical ways. I'm not an Apple zealot, but I won't ding them for not being able to support a closed source walled garden API.
- yza 6y agoSlightly off-topic, but I found M1 to be extremely fast at Gaussian Processes (using GPytorch/Botorch). I'm doing a PhD in Machine Learning for protein research, and for the heavy Bayesian Optimisation loop, the MBP M1 is almost twice as fast as my 16 core Ryzen 3950X "deep learning" machine (both use the CPU).