4 ms·
I just got Llama 3.1 GGUFs working on my Mac laptop with a new plugin for my LLM CLI tool: https://llm.datasette.io/ https://llm.datasette.io/ Here's informati
by simonw 2y ago
I just got Llama 3.1 GGUFs working on my Mac laptop with a new plugin for my LLM CLI tool: https://llm.datasette.io/ https://llm.datasette.io/
Here's information on the new plugin: https://simonwillison.net/2024/Jul/23/llm-gguf/ https://simonwillison.net/2024/Jul/23/llm-gguf/
Once you've installed LLM ("brew install llm" or "pipx install llm" or "pip install llm") you can try the new plugin like this:
llm install llm-gguf
llm gguf download-model \
https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
--alias llama-3.1-8b-instruct --alias l31i
llm -m l31i "five great names for a pet lemur"
This is using the GGUF version of Llama 3.1 8B Instruct from here: https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/tree/main https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-...
- didgeoridoo 2y agoSimon, when do you sleep?