I've wondered the same thing. I thought they would have to re-train it to respond differently to certain prompts -- aside from hardcoding responses to certain p

I've wondered the same thing. I thought they would have to re-train it to respond differently to certain prompts -- aside from hardcoding responses to certain phrases, however, from how creative the DAN prompts have had to get it seems like the censorship is more intelligent than that. I don't think it's the second answer, (adding pre-prompts before the user prompt), because when they do that, it's possible to hack it back to reading out the previous prompt - that's how we found out Bing was internally named Sydney. This would also reduce the number of tokens that could be input by the user. I'd love to know the answer to this too.