4 ms·

they responded to my tweet last year and said they didn't quantize the models.

• boroboro4 1y ago
  It's very hard to find right now but I'm sure they said they don't quantize KV cache, but their weights are in fp8.