3 ms·
Imagine a circular list (in however you want to construct that) that matches the input size for the model. The prompt is initially loaded at the start of the l
by shagie 4y ago
Imagine a circular list (in however you want to construct that) that matches the input size for the model.
The prompt is initially loaded at the start of the list and the model is run and produces high activation on a single output. That token output is then fed to the end of the input circular list and also added to the "this is what the model returned."
This process of running the model, getting the token output and sending one copy to the input list and one copy to the return string is repeated until the number of tokens generated hits a numeric limit or a token that represents the stop token is encountered.