3 ms·
4o is heavily quantized perhaps and runs on hardware that conceivably could be used for 3-4 years?
by htrp 2y ago
4o is heavily quantized perhaps and runs on hardware that conceivably could be used for 3-4 years?
- danielhanchen 2y agoFair point! Ye if it's 3-4 year amortization that's reasonable - quantization could be another reason why some benchmarks for 4o aren't that good vs pure float8 / float16, and also how they made it much faster