3 ms·
A 32 GB GPU costs > $5000, for that you get a whole Mac Mini M2 Pro (not as fast as NVidia, but sufficient for memory hungry LLMs)
by singularity2001 4y ago
A 32 GB GPU costs > $5000, for that you get a whole Mac Mini M2 Pro (not as fast as NVidia, but sufficient for memory hungry LLMs)
- deleted 4y ago[deleted]
- babypuncher 4y agoIt's also just a weird nonsense comparison. You buy a GPU from Nvidia and you just get a GPU. You buy a Mac from Apple and you get a whole computer that includes a GPU. To make a meaningful cost comparison we need to include an equivalent computer in which to put Nvidia's GPU. Then we get into what workloads you need the GPU for.
- wkat4242 4y agoHe mentions specifically that for machine learning its performance was underwhelming. Now he mentions tensor flow which probably has very different requirements than running an LLM. But the point about the toolchain is a good one IMO. Also, running an LLM is pretty niche for now since there are no open models yet (meta's is only an unofficial leak)