3 ms·
I guess it could be generalized to filling up the context window with any token, but just making sure none of the tokens repeat. An interesting twist could be
by helloplanets 8mo ago
I guess it could be generalized to filling up the context window with any token, but just making sure none of the tokens repeat.
An interesting twist could be making sure a specific token is an anagram of the token N tokens back. This could possibly measure how much a model can actually plan forwards.