2 ms·
llama.cpp does it already. You tell it how many layers to offload to GPU, and it runs remaining ones on CPU.
by skirmish 3y ago
llama.cpp does it already. You tell it how many layers to offload to GPU, and it runs remaining ones on CPU.
loading thread…