I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger mod

I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger models. Honestly, I haven't even seen GPT-3.5-turbo exhibit this behavior myself, although I am willing to believe it could happen. Llama 7B, however, goes off-script constantly.