3 ms·
You can run 25-30b model easily if you use Q3 or Q4 quants and llama-server with a pretty long list of options.
by Ladioss 6mo ago
You can run 25-30b model easily if you use Q3 or Q4 quants and llama-server with a pretty long list of options.
loading thread…