When GPT-3 was hot but unavailable, there was an online experiment where you could query an AI philosopher. Invisible to the user, the GPT-3 would first be prom

When GPT-3 was hot but unavailable, there was an online experiment where you could query an AI philosopher. Invisible to the user, the GPT-3 would first be prompted that it was a great philosopher, after which the user query would be appended. You didn't actually see any of this mind you, only the GPT-3 output. But, you could get all kinds of unanticipated outputs by hacking the prompt in similar ways as the OP. I even got it to output its secret prompting, which in turn helps to make new hacks. Ahh, fun times.