GPT4 is much better in that regard. With GPT3.5 I've had very similar problems. Plainly ignoring very simple directions (e.g. Do not output sentences starting w

GPT4 is much better in that regard. With GPT3.5 I've had very similar problems. Plainly ignoring very simple directions (e.g. Do not output sentences starting with "In conclusion"). GPT4 unfortunately still does that, but it's like 10% of the problem that it used to be. I also happened to test nested contexts, and at least when going meta-meta it's doing noticeably better than it did before. I would have never guessed such performance is achievable with this kind of token predicting text-only model.