4 ms·
The speed of GPT-4 via chatgpt varies greatly on when you’re using it. Could the data have been collected when the system is under different loads?
by bredren 3y ago
The speed of GPT-4 via chatgpt varies greatly on when you’re using it.
Could the data have been collected when the system is under different loads?
- pseudosavant 3y agoUnless they captured many different times and days, that is very likely a factor. GPU resources are constrained enough that during peak times (which vary across the globe) the token throughput will vary a lot.
- MacsHeadroom 3y agoThe speed data is an average over 30 days. Clearly OpenAI is throttling their API to save costs and get more out of fewer GPUs.