4 ms·

usually performance takes a hit with quantization. are you getting quality responses?

• rini17 2y ago
  Since llama3, yes, quite satisfying.