4 ms·

I think currently M2 max is best bang for buck running interface in open source model. But use case is so niche that Apple probably doesn't actively start suppo

I think currently M2 max is best bang for buck running interface in open source model. But use case is so niche that Apple probably doesn't actively start supporting open source models. In the long run I hope some smaller company gets shit together and starts competing with NVIDIA.

behnamoh 3y ago
At $6,000, how is M2 Max the best bang for the buck?! One could get two used 3090s and setup a decent PC at lower prices.
- kiratp 3y ago
  Two 3090s don’t have 96GB of VRAM
  - behnamoh 3y ago
    That's true. But OTOH, one can't easily upgrade Macs.
    - aarong11 3y ago
      With 96gb of VRAM, will you really need to? Personally I think apple servers might be adopted for AI and LLM workloads soon.
- antupis 3y ago
  M2 max studio is 2000$.
rankun203 3y ago
The GPU support in ML frameworks however is really not impressive. I have a Macbook with M1 Max 64G RAM, I can load a 7b model for fine-tuning (Huggingface Trainer, Pytorch, MPS), but the speed is just too slow, can only reach to 50% the speed of an i5-12500 CPU in my tests.