4 ms·
I got 700+ tokens/sec on o3 after the announcement, I suspect it's very much a quantized version. https://x.com/hyperknot/status/1932476190608036243 https://x.
by hyperknot 1y ago
I got 700+ tokens/sec on o3 after the announcement, I suspect it's very much a quantized version.
https://x.com/hyperknot/status/1932476190608036243 https://x.com/hyperknot/status/1932476190608036243
- dist-epoch 1y agoOr maybe they just brought online much faster much cheaper hardware.
- az226 1y agoOr they are using a speedy add-on decoder.
- zackangelo 1y agoIs that input tokens or output tokens/s?
- beering 1y agoDo you also have numbers on intelligence before and after?