3 ms·
Does anyone know if this works with llama.cpp?
by xrd 3y ago
Does anyone know if this works with llama.cpp?
- xrd 3y agoThere is an issue: https://github.com/ggerganov/llama.cpp/issues/2262 https://github.com/ggerganov/llama.cpp/issues/2262 But, short story seems to be: not yet.
- brucethemoose2 3y agoGGML quantizations are already being uploaded to huggingface, suggesting it works out of the box. GPTQ files are being uploaded too, meaning exLLaMA also might work.