19 ms·
I think the GPU claims are interesting. According to the graph's footer, the M1 Ultra was compared to an RTX 3090. If the performance/wattage claims are correct
by polyrand 5y ago
I think the GPU claims are interesting. According to the graph's footer, the M1 Ultra was compared to an RTX 3090. If the performance/wattage claims are correct, I'm wondering if the Mac Studio could become an "affordable" personal machine learning workstation (which also won't make the electricity bill skyrocket).
If Pytorch becomes stable and easy to use on Apple Silicon [0][1], it could be an appealing choice.
[0]: https://github.com/pytorch/pytorch/issues/47702#issuecomment-965625139 https://github.com/pytorch/pytorch/issues/47702#issuecomment...
[1]: https://nod.ai/pytorch-m1-max-gpu/ https://nod.ai/pytorch-m1-max-gpu/
- Koshkin 5y agoFor some definitions of "affordable."
- savant_penguin 5y agoAnd hopefully not make you deaf with their buzzing fans
- LegitShady 5y agoI always take such claims with a grain of salt anyways. It usualy on one specific benchmark. I wait for better benchmarks always instead of trusting the marketing
- pathartl 5y agoEven if their claims are accurate, it usually has the asterisk of *Only with Apple Metal 2. I honestly cannot understand why Apple decided they needed to write their own graphics API when the rest of the world is working hard to get away from the biggest proprietary graphics API.
- jpetso 5y agoI'd like to note that Microsoft is doing no better with DirectX. If it weren't for the drivers on Windows being distributed and supported by the GPU manufacturers themselves, Vulkan would only be a thing for Linux (incl. Android) and custom niche devices now.
- GekkePrutser 5y agoBecause vendor lock-in and full control over their APIs which has always been an apple staple but especially now. 5-10 years ago they were still serious about open standards, like OpenCL.. Now it's all locked in.
- saagarjha 5y agoBecause when they wrote the API no real standard existed to suit their needs.
- pathartl 5y agoYes, and instead of writing an open standard or adapting Vulkan when it came around, they instead decided to double down on proprietary and moved to Metal 2.
- saagarjha 5y agoI mean, Metal is a good API and they designed it first. They could've decided to abandon it when Vulkan came out, but I guess they just didn't want to make their API worse. Not ideal, yes, but I don't really blame them for the decision.
- whoisburbansky 5y agoCursory look gives you a ~$3500 price tag for a gaming PC with a 3090 [1], vs. at least $4k for a Mac Studio with an M1 Ultra. Roughly the same ballpark, but I wouldn't call the M1 Ultra more affordable given those numbers. 1. https://techguided.com/best-rtx-3090-gaming-pc/#:~:text=With%20this%20build%2C%20and%20an,for%20a%20little%20over%20%243%2C500 https://techguided.com/best-rtx-3090-gaming-pc/#:~:text=With....
- vimy 5y agoYou also need to compare the right cpu. M1 Ultra cpu is the equivalent of the fastest threadripper. Which costs $3990. So a pc with similar performance would be $7500
- cyber_kinetist 5y agoNot the top-of-the-line threadripper (which can go up to 32c/64t), but probably similar to the 5950x (16c/32t), which costs like 1000$. But you’re comparing apples to oranges, because the real advantage of M1 chips is the unified memory - almost no CPU-GPU communication overhead, and that the GPU can use ginormous amounts of memory.
- GekkePrutser 5y agoA threadripper also has many more PCIe lanes than a Ryzen. It's a bit of a different usecase I think although there's overlap.
- BugsJustFindMe 5y ago> Cursory look gives you a ~$3500 price tag for a gaming PC with a 3090 That 3500 is for a DIY build. So, sure, you can always save on labor and hassle, but prebuilt 3090 rigs commonly cost over 4k. And if you don't want to buy from Amazon because of their notorious history of mixing components from different suppliers and reselling used returns, oof, good luck even getting one.
- 5y ago
- kllrnohj 5y agoThe GPU claims on the M1 Pro & Max were, let's say, cherry picked to put it nicely. The M1 Ultra claims already look suspicious since the GPU graph tops out at ~120W & the CPU graph tops out at ~60W yet the M1 Studio is rated for 370W continuous power draw. Since you mention ML specifically, looking at some benchmarks out there (like https://tlkh.dev/benchmarking-the-apple-m1-max#heading-gpu https://tlkh.dev/benchmarking-the-apple-m1-max#heading-gpu & https://wandb.ai/tcapelle/apple_m1_pro/reports/Deep-Learning-on-the-M1-Pro-with-Apple-Silicon---VmlldzoxMjQ0NjY3 https://wandb.ai/tcapelle/apple_m1_pro/reports/Deep-Learning... ), even if the M1 Ultra is 2x the performance of the M1 Max (so perfect scaling), it would still be far behind the 3090. Like completely different ballpark behind. But of course there is that price & power gap, but the primary strength of the M1 GPUs seems to really be from the essentially very large VRAM amount. So if your working set doesn't fit in an RTX GPU of your desired budget, then the M1 is a good option. If, however, you're not VRAM limited, then Nvidia still offers far more performance. Well, assuming you can actually buy any of these, anyway. The M1 Ultra might win "by default" by simply being purchasable at all unlike pretty much every other GPU :/
- brigade 5y ago100W of that is probably for the USB ports; afaik TB4 ports are required to support 15W, and I don’t think there’s been a Mac didn’t support full power simultaneously across all ports. (if that’s even allowed?)
- dingle_thunk 5y agoI suppose given that this is two M1 Max glued together, assuming cooling is a solved problem, the max SOC power consumption just twice as high as usual, plus interconnect overhead. Right? Based on the thermal and power consumption characteristics of previous chips I would not be surprised if say ~120W is the max power draw of this thing. edit: Of course the M1 max only shipped in laptops, so... who knows.
- masklinn 5y ago> assuming cooling is a solved problem I’d assume that’s what most of the chonk is about, no? > Based on the thermal and power consumption characteristics of previous chips I would not be surprised if say ~120W is the max power draw of this thing. The Max could be brought up to 90W or so.
- forgotmyoldacc 5y agoNeural Engine cores are not accessible for third party developers, so it'll be severely constrained for practical purposes. Currently the M1 Max is no match for even last generation mid-tier Nvidia GPU.
- viktorcode 5y agoThey are accessible to third party developers, only they have to use CoreML.
- komuher 5y agoxD
- dev_tty01 5y agoHuh? Neural engine is certainly usable by developers. You just use the CoreML framework.
- alasdair_ 5y agoNote the label on the y-axis. "relative performance" from "0-200" seems like marketing bullshit to me. "M1 Ultra has a 64-core GPU, delivering faster performance than the highest-end PC GPU available, while using 200 fewer watts of power." Note that they say "faster performance" not "more performance". What does "faster" mean? Who knows!
- Retric 5y agoI have heard this argument before, if it’s identical workloads you get faster output but the same total work. Thus “faster performance” seems correct for fixed workloads and “more performance” is correct on games or benchmarks where you get more FPS. I still think “faster performance” sounds sound odd, but I understand their point.
- fomine3 5y agoAnyway any benchmark from first party should be taken with a mountain of salt.
- metalliqaz 5y agoI know Nvidia has never really care much about TDP, but this still seems unbelievable to me. How could a relatively new design beat a 3090 with 200W less power, while having to share a die with a CPU? It just doesn't seem possible.
- schmuelio 5y agoUnless the M1 Ultra is actually magic I don't think it is possible. My guess is they're putting a lot of weight on the phrase "relative power" and hoping you assume it means "relative to each other" and not "relative to their previous generation" (i.e. M1 Ultra -> M1 Max and RTX 3090 -> RTX 2080Ti) or "relative to the stock power profile". Put bluntly, if the M1 Ultra was capable of achieving performance parity with an RTX 3090 for any GPU-style benchmark then Nvidia (who are experts in making GPUs) would have captured this additional performance. Bear in mind the claim seems to be (on the surface) that the M1 Ultra is achieving with 64 GPU cores and 800GB/s memory bandwidth what the RTX 3090 is achieving with 10,496 GPU cores and 936.2GB/s memory bandwidth.
- kllrnohj 5y agoIt's actually kinda magic but in the opposite way. The M1 Ultra is absolutely massive. 114 billion transistors. The 3090? A measely 28 billion. Now granted the M1 also has some CPU cores on there but even still, it seems safe to say that the M1 Ultra has more transistors spent on the GPU than a 3090 does. More transistors does often mean more faster when it comes to GPUs. But you'll all but certainly see the 3090 win more benchmarks (and by a landslide) than the M1 Ultra does. Because Nvidia is really, really fucking good at this, and they spend an absurd amount of money working with external projects to fix their stuff. Like contributing a TensorFlow backend for CUDA. Or tons of optimizations in the driver to handle game-specific issues. Meanwhile Apple is mostly in the camp of "well we built Metal2, what's taking ya'll so long to port it to our tiny marketshare platform that historically had terrible GPU drivers?"
- schmuelio 5y agoGiven things like the AMD Epyc cores sit at around 40 billion transistors, I would assume a (roughly) even split. It is also worth noting that the M1 Ultra is an SoC so it'll have more than just CPU/GPU on it, by the looks of things it has some hefty amounts of cache, it'll also have a few IP blocks like a PCIe controller, memory controller, SSD controller (the current "SSDs" look to just be raw storage modules). All told it likely still has somewhere in the region of 30-40 billion transistors for the GPU. Each GPU core being physically bigger than the 3090 is probably pretty good for some workflows and not so good for others. Generally GPUs benefit from having a huge number of tiny cores for processing in parallel, rather than a small number of massive cores. Current benchmarks put it at roughly the performance of an RTX 3070, which is good for its power consumption, but not even close to the 3090. As I mentioned in the previous post, it just doesn't have the cores or memory bandwidth needed for the types of workloads that GPUs are built for (although unified memory being physically closer can help here ofc.), certainly not enough to make it a competitor for something like a 3090. Edit: Oh also, for massively parallel workloads (like what GPUs do), more cores and better bandwidth to feed those cores will be one of the biggest performance drivers. You can get more performance by making those cores bigger (and therefore faster) but you need to crank the transistor count up a _lot_ to match the kinds of throughput that many tiny cores can do.
- upbeat_general 5y agoI'm very skeptical of this because until the CUDA strangehold is gone, it will be a pain to develop on. Even if the frameworks themselves support M1's GPU, there are still lots and lots of CUDA kernels that won't run. I really hope I'm wrong (as someone who owns an M1 Pro chip) but I find it hard to imagine things changing significantly in the next ~2 years unless someone is able (legally and technically) to release a CUDA compatibility layer.
- jpetso 5y agoHere's AMD's attempt: https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY Naturally, the HIP tooling doesn't support M1 GPUs at this time. We'll see if anyone else tries.
- TuringNYC 5y agoIt shocks me how much payroll and cap-ex is spent on the M1 and how little is invested in getting TensorFlow/Pytorch to work on it. I could 10x my M1 purchases for our business if we could reliably run TensorFlow on it. Seems pretty shortsighted. The GPU claims wouldnt even need to be on parity with NVIDIA, it would just need to offer a vertically integrated alternative to having to use EC2.
- moflome 5y ago> reliably run TensorFlow What reliability issues are you having with TensorFlow on M1 Macs?
- TuringNYC 5y agoWe've followed five different instructional and documentation pages to make it happen and none seem to consistently install. Throw in a corporate system where you need IT for root access to make changes and it is game over. So i've got an M1-max fully loaded and cant get TF running on it. Now i've got a team of data scientists in a fully MBP shop and we're holding off upgrades to M1 until this all gets resolved. On my personal M1, I managed to make it work, but its hard to know the layers of changes made and what exactly allowed it to work.
- fouc 5y agoCould be worth tracking what you did and make a new set of instructions, and trying to reproduce with a fresh install.
- freemint 5y agoThis is something Apple should pay people for.
- hedgehog 5y agoDeep learning support for Mac is not going to happen at a level of quality you can rely on for research & dev work (like PyTorch + TensorFlow). The underlying problem is no big company cares about Mac platform and the work to maintain framework support for a specific piece of hardware is way beyond a hobby project. If you want your own on-prem hardware just buy Nvidia.
- hyperpallium2 5y agoDoesn't "electricity bill" dominate server/DL/nining datacenter workload costs these days - so perf/W is what really counts? The long-prophecised ARM apocalypse may finally be at hand.
- sytelus 5y agoThe most important detail here is 128GB ram for GPU computation! This allows to train monster models, ex 1B params GPT series, on single M1 Ultra. This is quite unprecedented. Unfortunately, it is also about 3.5x slower than 3090.
- rasz 5y agoApple loves to compare incomparable stuff. G5 the "world's fastest personal computer" etc. Its easy to claim GPU performance when you dont support modern OpenGL nor Vulcan so nobody can just run modern games and verify and you end up with "relative performance" graph whatever that means.
- izacus 5y agoThe GPU claims for M1 Pro and M1 Max were wildly above their actual performance in real life (as opposed to CPU performance) so maybe don't put all that much faith in Apple marketing here either.
- gameswithgo 5y agoGPU claims at carefully chosen tasks that don't need GDDR6 to get top performance. It wouldn't game like a 3090 for instance, but then not many people are gaming on macs anyway.