3 ms·
We used to just show the answer back when our models were 10x smaller than they are now. With scale, which yields higher quality answers, comes slower speed. He
by rushingcreek 4y ago
We used to just show the answer back when our models were 10x smaller than they are now. With scale, which yields higher quality answers, comes slower speed. Hence streaming the text is the compromise.
- StephenSmith 4y agoCool, thanks for the response. Have you talked to a UI person about this? I wonder if it would be better to load it in chunks? The rolling text might be considered distracting. Though, I'm no expert.