I suppose in some cases it could educate people on how to do bad things well enough to be dangerous. Otherwise, as GPT becomes more sophisticated and reliably

I suppose in some cases it could educate people on how to do bad things well enough to be dangerous. Otherwise, as GPT becomes more sophisticated and reliably correct, jail breaks will have more profound implications. Finding holes early is important both for ensuring it’s patched before it becomes more dangerous, but also interesting for revealing more of its capabilities in the meantime. It isn’t clear how much it’s guard rails restrain it’s abilities at this point. As far as security, I’m not sure it could expose enough about the implementation that’s not already in the paper. I suspect it’s more of a concern that people will try to use it for nefarious things, and they might succeed more than they would without this tool.