3 ms·
You can already run these models locally with Ollama (ollama run llama3.1:latest) along with at places like huggingface, groq etc. If you want a playground to
by ajhai 2y ago
You can already run these models locally with Ollama (ollama run llama3.1:latest) along with at places like huggingface, groq etc.
If you want a playground to test this model locally or want to quickly build some applications with it, you can try LLMStack (https://github.com/trypromptly/LLMStack https://github.com/trypromptly/LLMStack). I wrote last week about how to configure and use Ollama with LLMStack at https://docs.trypromptly.com/guides/using-llama3-with-ollama https://docs.trypromptly.com/guides/using-llama3-with-ollama.
Disclaimer: I'm the maintainer of LLMStack
- jxy 2y agoYou are a maintainer of a software that depends on ollama, so you should know that ollama depends on llama.cpp. And as of now, llama.cpp doesn't support the new ROPE: https://github.com/ggerganov/llama.cpp/issues/8650 https://github.com/ggerganov/llama.cpp/issues/8650, and all ollama can do is wait for llama.cpp: https://github.com/ollama/ollama/issues/5881 https://github.com/ollama/ollama/issues/5881
- ajhai 2y agoI've tested Q4 on M1 and it works though the quality may not likely be the same as you'd expect as others have pointed out on the issue.