3 ms·
Looks like no quantized options with llama.cpp? https://github.com/ggerganov/llama.cpp/issues/1602 https://github.com/ggerganov/llama.cpp/issues/1602
by xrd 3y ago
Looks like no quantized options with llama.cpp?
https://github.com/ggerganov/llama.cpp/issues/1602 https://github.com/ggerganov/llama.cpp/issues/1602
- dvilasuero 3y agoWe're very much looking forward to seeing Falcon-40B support on llama.cpp. For production use cases, this is also highly relevant: https://huggingface.co/blog/sagemaker-huggingface-llm https://huggingface.co/blog/sagemaker-huggingface-llm