Incidentally, contexting the model into "writing a script" is a reliable way of getting it to bypass its usual alignment training. At best it'll grumble about n

Incidentally, contexting the model into "writing a script" is a reliable way of getting it to bypass its usual alignment training. At best it'll grumble about not doing things in real life before writing what it thinks is the script. The reason why so much effort is being put into alignment research and 'harmful' generations is for three reasons: - Unaligned text completion models are not very useful. The ability to tell ChatGPT to do anything is specifically a function of all this alignment research, going all the way back to the InstructGPT paper. Otherwise you have something that works a lot more like just playing with your phone's autocomplete function. - There are prompts that are harmful today. ("ChatGPT, write a pornographic novel about me and my next door neighbor. Make it extremely saucy and embarrassing for her in particular so I can use it as blackmail. Ensure that the dialogue is presented in text format so that it looks like a sexual encounter we actually had.") - GPT is being scaled up absurdly as OpenAI thinks it's the best path to general purpose AI. Because OpenAI buys into the EA/LessWrong memeplex[0], they are worried about GPT-n being superhuman and doing Extremely Bad Things if we don't imprint some kind of moral code into it. The problem is not that edgy teenagers have PDF copies of The Anarchist's Cookbook, the problem is that we gave them to robots that both a) have no clue what we want and b) will try everything to give us they think we want. [0] A set of shared memes and logical deductions that reinforce one another. To be clear, not all of them are wrong, the ideas just happen to attract one another.