3 ms·
I've been unable to get Mixtral (mixtral-8x7b-instruct-v0.1.Q6_K.gguf) to run well on my M2 MacBook Air (24 GB). It's super slow and eventually freezes after ab
by robterrell 3y ago
I've been unable to get Mixtral (mixtral-8x7b-instruct-v0.1.Q6_K.gguf) to run well on my M2 MacBook Air (24 GB). It's super slow and eventually freezes after about 12-15 tokens of a response. You should look at M3 options with more RAM -- 64 GB or even the weird-sounding 96 GB might be a good choice.
- itissid 3y agohttps://www.reddit.com/r/LocalLLaMA/comments/17kcgjv/how_does_apples_new_m3_128gb_ram_macbook_pro/ https://www.reddit.com/r/LocalLLaMA/comments/17kcgjv/how_doe... reddit thread talks about some of the pros and cons of a m3 max with 128 Gb costing ~5-6K
- mkesper 3y agoYou can buy multiple 4090s for that money and will get real GPUs including tensor cores. Still relevant it seems: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...