Practically as the adult in charge you have to recognize when this is going on. With Junie and other IDE-based coding agents my experience is that sometimes th

Practically as the adult in charge you have to recognize when this is going on. With Junie and other IDE-based coding agents my experience is that sometimes the context goes bad and once that happens the best thing to do is start a new session. If you ask it to do something and it gets it 80% right and then you say "that's pretty good but..." and it keeps improving that's great... But once it doesn't seem to be listening to you or is going in circles or you feel like you are arguing it is time to regroup. Negation is one of the hardest problems in logic and NLP, you're better off explaining what to do instead of saying "DO NOT ..." as the attention mechanism is just as capable of locking on to the part after the DO NOT as it is on locking onto the whole thing. Reasoning with uncertainty is another super-hard problem, I tend to think the "language instinct" is actually a derangement about reasoning about probabilities that cause people to make the same mistakes and collapse the manifold of meanings to a low-dimensional space that is learnable... LLMs work because they make the same mistakes too. Circa 2018 I was working for a startup that was trying to develop foundation models and I was the pessimist who used a method of "predictive evaluation" who could prove that "roughly 10% of the time the system loses some critical information for making a decision and that gives an upper limit of 90% accuracy" which was right in the sense that I was thinking like a math teacher who rejects "getting the right answer by the wrong means" but wrong in the sense that people might not care about the means and be happy to get 95% accuracy if it guesses right half the time. My thinking was never going to lead to ChatGPT because I wasn't going to accept short circuiting.