3 ms·
> For people without a 24GB RAM video card, I've got an 8GB RAM one running What're you using for this? llama.cpp? Have a 12GB card (rtx 4070) i'd like to try
by throwaway314155 1y ago
> For people without a 24GB RAM video card, I've got an 8GB RAM one running
What're you using for this? llama.cpp? Have a 12GB card (rtx 4070) i'd like to try it on.
- johnQdeveloper 1y agohttps://ollama.com/library/devstral https://ollama.com/library/devstral https://ollama.com/ https://ollama.com/ I believe its just a HTTP wrapper and terminal wrapper around llama.cpp with some modifications/fork.
- throwaway314155 1y agoDoes ollama have support for cpu offloading?
- johnQdeveloper 1y ago> Does ollama have support for cpu offloading? https://www.reddit.com/r/ollama/comments/1df757o/high_cost_of_cpu_offloading/ https://www.reddit.com/r/ollama/comments/1df757o/high_cost_o... https://github.com/ollama/ollama/issues/8291 https://github.com/ollama/ollama/issues/8291 Yes.
- taneq 1y agoA perfect blend of LMGTFY and helpfulness. :)
- johnQdeveloper 1y agolol. I try not to be a total asshole, it sometime even works! :) Good luck to you mate with your life :)