3 ms·
Ah so you do! Your latency numbers for OpenAI (and Azure's equivalents) seem really high, I run time to first token tests and I see much better numbers! (Also
by com2kid 3y ago
Ah so you do!
Your latency numbers for OpenAI (and Azure's equivalents) seem really high, I run time to first token tests and I see much better numbers!
(Also are those numbers average, p50, p99, etc? I'd honestly expect a box plot to really see what is going on!)
- Gcam 3y agoHey com2kid - if you're still there, we did end up adding boxplots to show variance. Can be seen on the models page https://artificialanalysis.ai/models https://artificialanalysis.ai/models and on each models page where you view hosts by clicking one of the models. They are toward the end of the page under 'Detailed performance metrics'