3 ms·
I think the general (completely speculative and unconfirmed) consensus is that "Turbo" models are somehow quantized, or otherwise modified for much faster and c
by andy_xor_andrew 3y ago
I think the general (completely speculative and unconfirmed) consensus is that "Turbo" models are somehow quantized, or otherwise modified for much faster and cheaper inference, at the expense of some quality (the definition of "some" is unknown here).
well, that's how it was for GPT-3.5 anyway. The "turbo" flavor was faster and cheaper, but seemed to have slightly worse output (again, this is all going by subjective measurement; it could entirely be the imagination of AI bros)
- xeckr 3y agoI agree that this is the case for GPT-3.5, at least subjectively. However, with GPT-4 Turbo, it seems that performance has improved. If they got it to be faster using quantization, then they must have also found a way to offset any resulting performance losses.
- letitgo12345 3y agoThey have tons of usage data by now to figure out which queries to devote model capacity to