Hang on I think you're assuming there's some internal state that I don't think exists. Each time it just generates one more token until it hits a stop token, u

Hang on I think you're assuming there's some internal state that I don't think exists. Each time it just generates one more token until it hits a stop token, using the previous input. It shouldn't matter if it created a token or you did.