3 ms·
> (1) Is there such thing as a "end of reply" token ? (I'm not taking about End of Sentence <EOS> which is obviously different) EOS as end of sentence? IIRC it
by refulgentis 2y ago
> (1) Is there such thing as a "end of reply" token ? (I'm not taking about End of Sentence <EOS> which is obviously different)
EOS as end of sentence? IIRC its end of sequence, which may also answer the question IIUC.
- sidkshatriya 2y ago<EOS> is end of sequence ! that makes sense, thanks for the correction. So, if we simply wait for <EOS> to be sampled in a block, that could be the place to stop. My answer above should be the same (modulo this correction) -- simply keep obtaining the blocks in a semi-autoregressive fashion.