3 ms·
But (in theory) - llama.cpp could implement similar approach to paging/memory and see a speedup for 4bit models on cpu?
by e12e 3y ago
But (in theory) - llama.cpp could implement similar approach to paging/memory and see a speedup for 4bit models on cpu?