3 ms·
It can run larger models quite slowly but lacks matmul acceleration (included in the M5) that is very useful for context and prompt performance at inference tim
by andy_ppp 1y ago
It can run larger models quite slowly but lacks matmul acceleration (included in the M5) that is very useful for context and prompt performance at inference time. I will probably burn my budget with an M5 Max with 256gb (maybe even 512gb) memory, the price will be upsetting but I guess that is life!
- sgt 1y agoYes! I think smaller models on the M3 Ultra is interesting enough, but now with matmul/ tensors on M5 Ultra or Max, with decent unified mem, it will be a gamechanger. I can easily imagine companies running Mac Studios in prod. Apple should release another Xserve.
- andy_ppp 1y agoYes completely, my guess is M6 will have external GPUs perfect for AI accelerators at home and in datacenters.