I think you are missing something, or I am. If we observe the performance comparison graph (1), 2.8->9 tok/s is achieved via quantization, but the remaining ju

I think you are missing something, or I am. If we observe the performance comparison graph (1), 2.8->9 tok/s is achieved via quantization, but the remaining jumps from 9->16.6->24.6 tok/s are achieved from the sparse fine-tuning. (1) https://neuralmagic.com/wp-content/uploads/2023/11/CHART-Llama2-Sparse_Fine_tuned_GSM8k_Text_Generation_Performance-005-1-1536x1000.png https://neuralmagic.com/wp-content/uploads/2023/11/CHART-Lla...