3 ms·
I think it's heavily quantized, so it doesn't cost them (too much). But I think it's still at cost...
by yawnxyz 2y ago
I think it's heavily quantized, so it doesn't cost them (too much). But I think it's still at cost...
- saiansh2525 2y agoJudging from the perplexity scores, the model doesn't seem to be quantized, it seems to simply be a scaled down version of the original GPT-4O or something similar.