3 ms·

I’ve been running Orca 2 13B on M1 Pro with 32GB of RAM with LLM Studio and GPU acceleration quite nicely. https://huggingface.co/TheBloke/Orca-2-13B-GGUF http

I’ve been running Orca 2 13B on M1 Pro with 32GB of RAM with LLM Studio and GPU acceleration quite nicely. https://huggingface.co/TheBloke/Orca-2-13B-GGUF https://huggingface.co/TheBloke/Orca-2-13B-GGUF

nicbou 3y ago
Ollama runs 13b models just fine on my M2 Air with 16 GB
evnc 3y ago
What kind of tokens/sec do you get?