3 ms·
...and quantized ones from the usual suspect: https://huggingface.co/TheBloke/Orca-2-7B-GGUF https://huggingface.co/TheBloke/Orca-2-7B-GGUF https://huggingfac
by T-A 3y ago
...and quantized ones from the usual suspect:
https://huggingface.co/TheBloke/Orca-2-7B-GGUF https://huggingface.co/TheBloke/Orca-2-7B-GGUF
https://huggingface.co/TheBloke/Orca-2-13B-GGUF https://huggingface.co/TheBloke/Orca-2-13B-GGUF
The 7B Q5_K_M one is small enough to run on an 8GB consumer GPU.
- ganeshkrishnan 3y agoAll the 13B files seems to be quantized.