2 ms·
What I find interesting is that, with some complicated tasks, the model will produce output that reiterates some of the key points from earlier, and it can cont
by int_19h 4y ago
What I find interesting is that, with some complicated tasks, the model will produce output that reiterates some of the key points from earlier, and it can continue to do so for a very long time (i.e. long enough that they'd normally be out of the token window). It feels vaguely like crude short-term memory.