3 ms·
It feels less hamstrung on output, but is instead hamstrung by being incredibly slow. GPT-4 via API will sometimes take 30 seconds + to respond to simple quest
by jonplackett 3y ago
It feels less hamstrung on output, but is instead hamstrung by being incredibly slow.
GPT-4 via API will sometimes take 30 seconds + to respond to simple questions without any chat history, where through Chat GPT it will give you near enough instant replies only slightly slower than 3.5 turbo.
- danielbln 3y agoMaybe the reverse is true? To provide faster performance they dumb down gpt4 (quantization?) that serves the chatgpt UI but the API is full power albeit slow.
- jonplackett 3y agoI think it’s just prioritising ChatGPT. The reason that’s my guess is that the API is occasionally much faster and then goes slow again. It’s a bit all over the place which leads me to suspect it’s based on demand