As far as I know, GPT does the work one token at a time. It produces a single token at each iteration. So if you give it "abc" and want "def" as the result, you

As far as I know, GPT does the work one token at a time. It produces a single token at each iteration. So if you give it "abc" and want "def" as the result, you first give it "abc" and it returns "d", then you give it "abcd" and it returns "e", then you give it "abcde" and it returns "f", then you give it "abcdef" and it returns the stop token. So just by outputting its work, it can provide its own context to memoize what the answer for each bit is.