4 ms·
Does it really generate the text like that, little by little, or is that just a needless animation?
by lzooz 4y ago
Does it really generate the text like that, little by little, or is that just a needless animation?
- jw1224 4y agoIt's literally the model's output, in real-time. The GPT-3 API lets you turn streaming on or off, but if you don't enable it you have to wait much longer for the completion to finish entirely, before getting anything back.
- satvikpendem 4y agoYes because it's the way the AI's architecture works, it generates the next token in a phrase based on the previous tokens. You can see it happen in real time or you can wait until it's all finished.