2 ms·
Doesn't the 3090 cap out at 24GB VRAM? That's not a lot to run a local model
by kylec 3mo ago
Doesn't the 3090 cap out at 24GB VRAM? That's not a lot to run a local model
- mzubairtahir 3mo agobut still it can run handsome models
- EagnaIonat 3mo agoLatest MBP goes up to 128GB of memory.
- bigyabai 3mo agoThe M5 Max's GPU is slower than a single RTX 3090, though. Both machines will be stuck in the 30-50GB model range, but the 3090 would have faster token prefill and faster decode speeds (614GB/s on M5 Max vs 936GB/s on 1x 3090).
- EagnaIonat 3mo ago> Both machines will be stuck in the 30-50GB model range, That is not true. 3090 is only 24GB. It will absolutely outperform the M5 on models that will fit in that space, but M5 will outperform with larger models as the whole 128GB is GPU addressable.
- brcmthrowaway 3mo agohttps://old.reddit.com/r/LocalLLM/comments/1ug428c/4x3090_192gb_ddr5_9900x_running_llm/ https://old.reddit.com/r/LocalLLM/comments/1ug428c/4x3090_19...