4 ms·
There's gguf versions (for llama.cpp) at https://huggingface.co/TheBloke/Yarn-Mistral-7B-128k-GGUF https://huggingface.co/TheBloke/Yarn-Mistral-7B-128k-GGUF Ll
by kristianp 3y ago
There's gguf versions (for llama.cpp) at https://huggingface.co/TheBloke/Yarn-Mistral-7B-128k-GGUF https://huggingface.co/TheBloke/Yarn-Mistral-7B-128k-GGUF
Llama.cpp is good for running on macs and for partial gpu offloading on non-macs.
There's also a table of memory usage depending on which file you use.