The problem with retaining context is that it gets polluted. That pollution gets you into a latent space with errors, which probably not where you want your nex

The problem with retaining context is that it gets polluted. That pollution gets you into a latent space with errors, which probably not where you want your next token prediction to be sourced. The reasonable alternative is a chat interface that lets you edit any text, the AI response or your prompts, and regenerate from any point. This is why I use the API "playground" interfaces or something like LibreChat. Deepseek at least has prompt editing/regeneration.