4 ms·
Arguably it makes more sense technically to get the model support into llama.cpp, which provides many options for GPU+CPU split inference already.
by zozbot234 6mo ago
Arguably it makes more sense technically to get the model support into llama.cpp, which provides many options for GPU+CPU split inference already.