3 ms·

> I ran all the open models (anything not from OpenAI, meaning anything that doesn’t start with gpt or o1) myself using Q5_K_M quantization, whatever that is.

> I ran all the open models (anything not from OpenAI, meaning anything that doesn’t start with gpt or o1) myself using Q5_K_M quantization, whatever that is. It's just a lossy compression of all of the parameters, probably not important, right?

loa_in_ 2y ago
Probably important when competing against undecimated ones from OpenAI
- NiloCK 2y ago
  Notably: there were other OpenAI models that weren't quantize, but also performed poorly.