3 ms·
Will this run on an M3 48GB?
by isaacrolandson 2y ago
Will this run on an M3 48GB?
- piskov 2y agoYou’ll need 44GB just for the weights By default only 75% of unified memory is available to GPU if you have >36GB. So with 48 total only 36 is available for GPU with is lower than 44. tldr; without quantization you will not be able to run it.
- stalked_why 2y agohttps://medium.com/@tedisaacs/codestral-for-ai-coding-models-size-matters-3efcf9f4f856 https://medium.com/@tedisaacs/codestral-for-ai-coding-models... This article says 88GB without quantization. Though it then goes on to make a ridiculous claim that if you had 128GB of RAM, then using up 88GB of 128GB would make everything else really slow because I guess the think the remaining 40GB of RAM somehow isn't enough for your OS and desktop apps. So it’s probably not a very authoritative source.