3 ms·
> It's certainly not on a level with ChatGPT, but what is? Guanaco-65B is per https://arxiv.org/abs/2305.14314 https://arxiv.org/abs/2305.14314 CPU Version:
by MacsHeadroom 3y ago
> It's certainly not on a level with ChatGPT, but what is?
Guanaco-65B is per https://arxiv.org/abs/2305.14314 https://arxiv.org/abs/2305.14314
CPU Version: https://huggingface.co/TheBloke/guanaco-65B-GGML https://huggingface.co/TheBloke/guanaco-65B-GGML
GPU Version: https://huggingface.co/TheBloke/guanaco-65B-HF https://huggingface.co/TheBloke/guanaco-65B-HF
4bit GPU Version: https://huggingface.co/TheBloke/guanaco-65B-GPTQ https://huggingface.co/TheBloke/guanaco-65B-GPTQ
- VadimPR 3y agoHow much ram/vram required?
- MacsHeadroom 3y agoAt least 48GB of RAM to run with llama.cpp, which now has CUDA GPU Offloading support. The more you can fit on your GPU (in VRAM) the better (for speed), but no GPU is strictly required.
- balu548411 3y agohow can we load this 4bit GPU Version: https://huggingface.co/TheBloke/guanaco-65B-GPTQas https://huggingface.co/TheBloke/guanaco-65B-GPTQas that is in safetensors format