3 ms·
Looks like 109B would fit in a 64GiB machine's RAM at 4-bit quantization. Looking forward to trying this.
by api 2y ago
Looks like 109B would fit in a 64GiB machine's RAM at 4-bit quantization. Looking forward to trying this.
- tarruda 2y agoI read somewhere that ryzen AI 370 chip can run gemma 3 14b at 7 tokens/second, so I would expect the performance to be somewhere in that range for llama 4 scout with 17b active