3 ms·
Do LLM's generate words on the fly or can they sort of "go back" and correct themselves? stackghost brought up a good point I didn't think about before
by sigbottle 2y ago
Do LLM's generate words on the fly or can they sort of "go back" and correct themselves? stackghost brought up a good point I didn't think about before
- YZF 2y agoafaik they do not go back. keep in mind there is a context in which they are generating the response, e.g. the system prompt and the actual question.
- benlivengood 2y agoBeam search generates multiple potential completions and scores multiple tokens by likelihood, the picks the most likely after some threshold or length, which is close to a "go back and try again".