3 ms·
What's a realistic way to run this locally or a single expensive remote dev machine (in a vm, not through API calls)?
by fooker 7mo ago
What's a realistic way to run this locally or a single expensive remote dev machine (in a vm, not through API calls)?
- matja 7mo agoI'm running Gemma 4 with the llama.cpp web UI. https://unsloth.ai/docs/models/gemma-4 https://unsloth.ai/docs/models/gemma-4 > Gemma 4 GGUFs > "Use this model" > llama.cpp > llama-server -hf unsloth/gemma-4-31B-it-GGUF:Q8_0 If you already have llama.cpp you might need to update it to support Gemma 4.
- deleted 7mo ago[deleted]