4 ms·
Seems promising. Do they say anywhere how many tokens / second is achieved on the CPU?
by tarruda 3y ago
Seems promising. Do they say anywhere how many tokens / second is achieved on the CPU?
- littlestymaar 3y agoThey do. It's on the third graph: https://neuralmagic.com/wp-content/uploads/2023/11/CHART-Llama2-Sparse_Fine_tuned_GSM8k_Text_Generation_Performance-005-1-1536x1000.png https://neuralmagic.com/wp-content/uploads/2023/11/CHART-Lla...