3 ms·
Obviously nobody is competing with NVIDIA right now given their strong CUDA+Mellanox advantage. But that second place is a very important spot. I think if Appl
by dagmx 2y ago
Obviously nobody is competing with NVIDIA right now given their strong CUDA+Mellanox advantage. But that second place is a very important spot.
I think if Apple sold to businesses, it would be really bad news for AMD.
For comparison, take an MI300A which is AMDs H100 competitor. It’s under half the price ($20k) of an H100 (~$30-50k) with more HB3 memory. It maxes out at something like 128GB of memory. I can’t find the price of a GH100 in todays market.
Then there’s Apple. An M2 Ultra Mac Studio is <$7k for 192GB of memory. Granted it’s much slower memory, but that’s not a huge factor for many workloads.
So this M2 Ultra machine is 14% of the H100 and 35% of the MI300A. It has more memory and a faster per-core processor that consumes 40% of the power of the MI300A.
Now of course there aren’t good benchmarks to compare the exact loadouts and configurations here, and like I mentioned the memory is slower LPDDR5 instead of HBM3.
but even then, you can get multiple Mac Studios for the closest competitor. It’ll cost you significantly less to run. The extreme memory bandwidth of HBM3 is not unique to anyone and the price difference would likely more than cover it.
So then there’s the software moat. NVIDIA has CUDA and then after them, Apple seemingly has the widest support of ML libraries. AMD has improved a lot with Rocm but they’re still lagging behind Apple and way behind NVIDIA.
So anyway, if Apple decides to bring this to market, I would be very worried for anyone vying for second place. The biggest issues will be whether macOS is a turnoff (doubt it) and whether Apple can do enterprise (remains to be seen).
- google234123 2y agoTensorflow seems to run fine on Google TPUs.
- dagmx 2y agoYes but Google don’t sell their hardware to others, so there’s no pricing information to compare it with.
- edward28 2y agoThe mi300x is AMDs h100/200 competitor.
- dagmx 2y agoI know. But the M2 Ultra is the MI300A’s competitor. I couldn’t find the pricing for the GH100, but used the H100 as a standin since it’s already so much more expensive to not really affect the point I was making.
- seydor 2y agoDoes Apple ever compete on price?
- dagmx 2y agoYes but only if you shop to match the same criteria. E.g a MacBook Pro may seem overpriced if you compare to average laptop costs but if you shop within laptops with the same display, battery life and performance unplugged, it is very fairly priced. Similarly, Vision Pro seems expensive compared to average consumer headsets, but if you match its capabilities (eye tracking, high resolution display, etc) the next competitor are multiples of its price.
- Wowfunhappy 2y ago> The biggest issues will be whether macOS is a turnoff Could Asahi Linux become relevant here?
- dagmx 2y agoI would think not. The GPU is a bit part of the equation for compute , and there’s very little in the way of ML libraries targeting the standard graphics APIs available on Linux.
- talldayo 2y agoIs that true? Linux is the number one choice for training and inferencing deployment, somebody must be doing something right. It's not like OpenAI was training their LLMs using CoreML, Apple isn't even a contender in the game. They were late to supporting basic industry-standard libraries like Tensorflow and Pytorch, and they still don't commit to an Open, cross-platform alternative. Apple is the #1 loser from the lack of standardization here, and you're trying to make it out as a Linux problem. Imagine if someone from the industry could hear you talking right now.
- hpen 2y agoiOS developer here, CoreML is used for edge deployments in the ML space. Just not training. Obviously linux is king but that's for server side my dude.
- talldayo 2y ago"edge deployment" by which you mean client-side application-specific inference. That's not bad, but it's also table-stakes when ONNX/Tensorflow exists and obviously isn't analogous to the server products Nvidia is shipping today. You can "deploy" CoreML the same way Android developers deployed Tensorflow for edge compute 5 years ago, but that wasn't even interesting when people demoed it on Raspberry Pi. Apple is behind.
- Wowfunhappy 2y ago
- luyu_wu 2y agoIt's a tiny bit ignorant to even compare consumer memory to HBM. Anyone can get 1TB for their server for less than Ultra chips. The reason you go HBM is precisely because of bandwidth needs.
- dagmx 2y agoI specifically call out the memory difference though precisely because I know they’re different. But I do not believe the cost for 192GB of HBM3 + any related changes to the memory system on the chips would be $13k, which is the difference between the Mac Studio and the MI300A.