3 ms·
> When running larger models that don't fit into VRAM on macOS, Ollama will now split the model between GPU and CPU to maximize performance. Can anyone explain
by tosh 3y ago
> When running larger models that don't fit into VRAM on macOS, Ollama will now split the model between GPU and CPU to maximize performance.
Can anyone explain what this means? I thought Apple Silicon Macs have unified memory. Or is this only related to macOS running on Intel?
- brandall10 3y agoThis might be referencing older Macs, though it also could be referencing the default allocation ceiling of ~2/3 unified for GPU usage on Apple silicon. That said, the latter issue can be expanded to a higher hard limit so it's kinda moot.
- planetafro 3y ago...uh, everyone doesn't run a Mac? I run adn Nvidia GPU in Win11/WSL, so this should prove helpful in a running models that push my VRAM limits.
- whartung 3y agoDoes this version use the GPU on recent Intel Macs? I know folks have been looking into it, but don’t know if it works yet. As is, it works but the performance doesn’t make in particularly usable on the last model of the Intel iMac.