3 ms·Large transformer models can look back about 4000 tokens which is about 3000 words.by sharemywin 4y agoLarge transformer models can look back about 4000 tokens which is about 3000 words.