3 ms·
"The 1.58bit quantization should fit in 160GB of VRAM for fast inference" instruction for llama.cpp: https://huggingface.co/unsloth/DeepSeek-R1-GGUF#instructio
by homarp 2y ago
"The 1.58bit quantization should fit in 160GB of VRAM for fast inference"
instruction for llama.cpp:
https://huggingface.co/unsloth/DeepSeek-R1-GGUF#instructions-to-run-this-model-in-llamacpp https://huggingface.co/unsloth/DeepSeek-R1-GGUF#instructions...