3 ms·
Macbooks have pretty bad GPU per USD performance compared to Nvidia but this post is just a clickbait. "Why advertise some super AI chip if it doesn’t do AI st
by machinekob 4y ago
Macbooks have pretty bad GPU per USD performance compared to Nvidia but this post is just a clickbait.
"Why advertise some super AI chip if it doesn’t do AI stuff? Who knows." => Neural engine is fp16/int8 limited so it wouldnt be good enough for training anyway but its pretty good for inference with CoreML in a lot of cases. Also training anything outside Nvidia ecosystem is very bad cause software was written with CUDA first and everything else second (AMD for example have similar story to Metal backend).
As someone who works on Deep learning for past 6 years I can easily say CUDA wont be replaced anytime soon at least for training, almost every software out there was written for either CUDA or CPU.
And for local model development you don't need your rig to be powerful as training wont be done on your local machine anyway and MacBooks are good enough for that (a lot better than windows for sure).
- singularity2001 4y agoA 32 GB GPU costs > $5000, for that you get a whole Mac Mini M2 Pro (not as fast as NVidia, but sufficient for memory hungry LLMs)
- deleted 4y ago[deleted]
- babypuncher 4y agoIt's also just a weird nonsense comparison. You buy a GPU from Nvidia and you just get a GPU. You buy a Mac from Apple and you get a whole computer that includes a GPU. To make a meaningful cost comparison we need to include an equivalent computer in which to put Nvidia's GPU. Then we get into what workloads you need the GPU for.
- wkat4242 4y agoHe mentions specifically that for machine learning its performance was underwhelming. Now he mentions tensor flow which probably has very different requirements than running an LLM. But the point about the toolchain is a good one IMO. Also, running an LLM is pretty niche for now since there are no open models yet (meta's is only an unofficial leak)
- pathartl 4y agoHow is it a clickbait? He bought an M2 Mac based on the advertising and reports from other users. He did some pretty in-depth research into if the machine will work for his workflow. He decided it didn't because of a lack of performance for the price.