3 ms·
Swapping would be too slow. Llama.cpp allows you to specify that n layers load into the GPU's VRAM, and the rest load into main memory.
by rahimnathwani 3y ago
Swapping would be too slow.
Llama.cpp allows you to specify that n layers load into the GPU's VRAM, and the rest load into main memory.