Yes, caching the states of the sequence would make sense. An issue is that it's still more expensive to compute the new tokens even if you cache the states view

Yes, caching the states of the sequence would make sense. An issue is that it's still more expensive to compute the new tokens even if you cache the states viewed so far