3 ms·
I wish someone would also thoroughly measure prompt processing speeds across the major providers too. Output speeds are useful too, but more commonly measured.
by coder543 7mo ago
I wish someone would also thoroughly measure prompt processing speeds across the major providers too. Output speeds are useful too, but more commonly measured.
- JLO64 7mo agoIn my use case for small models I typically only generate a max of 100 tokens per API call, with the prompt processing taking up the majority of the wait time from the user perspective. I found OAI's models to be quite poor at this and made the switch to Anthropic's API just for this. I've found Haiku to be a pretty fast at PP, but would be willing to investigate using another provider if they offer faster speeds.
- asselinpaul 7mo agoOpenRouter has this information
- coder543 7mo agoI do not see prompt processing, only some kind of nebulous “throughput” that could be output or input+output, but definitely not input only.