3 ms·

Hi, we have this if you take a look at the models page (https://artificialanalysis.ai/models https://artificialanalysis.ai/models) and scroll down to 'Latency',

by Gcam 3y ago

Hi, we have this if you take a look at the models page (https://artificialanalysis.ai/models https://artificialanalysis.ai/models) and scroll down to 'Latency', and also on the API host comparison pages for each model (e.g. https://artificialanalysis.ai/models/llama-2-chat-70b https://artificialanalysis.ai/models/llama-2-chat-70b)

com2kid 3y ago
Ah so you do! Your latency numbers for OpenAI (and Azure's equivalents) seem really high, I run time to first token tests and I see much better numbers! (Also are those numbers average, p50, p99, etc? I'd honestly expect a box plot to really see what is going on!)
- Gcam 3y ago
  Hey com2kid - if you're still there, we did end up adding boxplots to show variance. Can be seen on the models page https://artificialanalysis.ai/models https://artificialanalysis.ai/models and on each models page where you view hosts by clicking one of the models. They are toward the end of the page under 'Detailed performance metrics'