2 ms·
Both versions are quantized and should use the same amount of RAM, the difference with QAT is the quantization happens during training time and it should result
by zorgmonkey 1y ago
Both versions are quantized and should use the same amount of RAM, the difference with QAT is the quantization happens during training time and it should result in slightly better (closer to the bf16 weights) output