2 ms·
Qwen 27B runs very comfortably on a 5090. You need to use Q4 quants and Q8 KV cache. Here's the math https://news.ycombinator.com/item?id=49514141 https://news
by redox99 26d ago
Qwen 27B runs very comfortably on a 5090. You need to use Q4 quants and Q8 KV cache. Here's the math
https://news.ycombinator.com/item?id=49514141 https://news.ycombinator.com/item?id=49514141