4 ms·
The Granite 4.1 3B model is only 2GB from Unsloth: https://huggingface.co/unsloth/granite-4.1-3b-GGUF https://huggingface.co/unsloth/granite-4.1-3b-GGUF I ran
by simonw 6mo ago
The Granite 4.1 3B model is only 2GB from Unsloth: https://huggingface.co/unsloth/granite-4.1-3b-GGUF https://huggingface.co/unsloth/granite-4.1-3b-GGUF
I ran it in LM Studio and got a pleasingly abstract pelican on a bicycle (genuinely not bad for a tiny 3B model - it can at least output valid SVG): https://gist.github.com/simonw/5f2df6093885a04c9573cf5756d34f59#file-granite-4-1-3b-svg https://gist.github.com/simonw/5f2df6093885a04c9573cf5756d34...
- tredre3 6mo agoDo you have any reasons to believe that granite is more immune to the effects of quantization than other tiny models? Otherwise it seems odd to judge a tiny model true capabilities by using its 4bit quant.
- simonw 6mo agoThis model is small enough that it might be sensible to try the same prompts against all of the quant sizes to try and spot any differences.