3 ms·
Where did you get that understanding? This doesn't really make any sense, how would GPT be able to stream token at a time in the first place?
by mewpmewp2 3y ago
Where did you get that understanding? This doesn't really make any sense, how would GPT be able to stream token at a time in the first place?
- razodactyl 3y agoThere's actually information provided during token generation that act as a level of confidence. You can definitely stream and choose the highest scoring values amongst a few shots at generating the best next token candidate.