Anything shitty that enters the context window shifts the entire thing to shittiness. So far this has been my experience on pretty much any model. Context feed

Anything shitty that enters the context window shifts the entire thing to shittiness. So far this has been my experience on pretty much any model. Context feeds on its output. Once you it goes that road, unless you stop it and give it enough counter examples and details of what you want (i.e. you're nudging it on latent space towards a better spot), it keeps degenerating. It gets even worse if the context window is compressed before you get a chance to correct. Long horizon agents can degenerate at machine speed. I still think you get much better results if you give them short horizon, well specified tasks.