3 ms·
> Firstly you don't need 64GB for that model. You might need that to run it with a longer context, KV cache size is a known issue with that model series.
by zozbot234 5mo ago
> Firstly you don't need 64GB for that model.
You might need that to run it with a longer context, KV cache size is a known issue with that model series.