I've seen it referred to as 'context drunk'. Imagine that you have your input to the context, 10000 tokens that are 99% correct. Each time the LLM replies it a

I've seen it referred to as 'context drunk'. Imagine that you have your input to the context, 10000 tokens that are 99% correct. Each time the LLM replies it adds 1000 tokens that are 90% correct. After some back-and-forth of you correcting the LLM, its context window is mostly its own backwash^Woutput. Worse, the error compounds because the 90% that is correct is just correct extrapolation of an argument about incorrect code, and because the LLM ranks more recent tokens as more important. The same problem also shows up in prose.