2 ms·
Backtracking, of sorts. The LLM descends a seemingly fruitful path, and all of a sudden that path seems less so. So you backtrack a few tokens ("erasing"/"undoi
by gspr 2mo ago
Backtracking, of sorts. The LLM descends a seemingly fruitful path, and all of a sudden that path seems less so. So you backtrack a few tokens ("erasing"/"undoing") and resample the probability distribution a bit back in the stream.
- krautsauer 2mo agoWould it retain some memory of the fact that it backtracked? That path becoming less likely somehow?
- gspr 2mo agoYeah that's what I was imagining. No idea how though, so this is just rambling on my part :)