4 ms·
> I was annoyed by the default OpenAI ChatGPT slow-typing animation I’d assumed ChatGPT’s slow-typing was actually the model’s real-time output, streamed to th
by jw1224 4y ago
> I was annoyed by the default OpenAI ChatGPT slow-typing animation
I’d assumed ChatGPT’s slow-typing was actually the model’s real-time output, streamed to the browser token-by-token. When I’ve made GPT3 API calls it’s worked like this, with streaming switched off I might need to wait ~20 seconds for a long response back.
I just tried your UI, and it’s so much faster! Was my assumption about real-time output wrong?
Great project, looking forward to using it more :)
- trungdq88 4y agoYep, I was surprised I about the response time too. I'm not sure what's going on under the hood, maybe OpenAI did something with the stream output that causes the slowness.
- lordnacho 4y agoPerhaps it is simply a way to limit the number of queries? If you force people to wait 10s for an answer they aren't writing new questions all the time.
- TeMPOraL 4y agoFor me it's been consistently the case that after 2-3 messages, the animation would start to visibly lag, and eventually (on 3rd or 4th reply) slow to a halt, sometimes mid-sentence, completely bricking the session. It's been like this since ChatGPT was first opened to public. I initially assumed it's just the usual case of a bloated SPA, but now I think maybe I'm being throttled on purpose?
- weird-eye-issue 4y agoIt uses a different model. ChatGPT Plus is fast too.
- IanCal 4y agoChatGPTs paid version is extremely fast, and this is using the paid API. The paid API feels slower than the paid UI, but this may be the difference between streaming results and getting everything in one payload.
- SomewhatLikely 4y agoI think the slow responses are the real generation rate when you time share the GPU. If you are willing to let GPUs occasionally sit idle though, you can probably get a faster generation rate by dedicating more of the GPU time to that one client.