2 ms·
Making Ai safe involves aligning it with the user. So that the ai produces outcomes in line with the users expectations. An ai that has been lobotomized will be
by irthomasthomas 2y ago
Making Ai safe involves aligning it with the user. So that the ai produces outcomes in line with the users expectations. An ai that has been lobotomized will be less likely to follow the users instructions, and, therefore, less safe.
I haven't read this article yet, but I read their last paper on super alignment.
I get the impression that they apply the lightest system prompts to chatgpt to steer it towards not answering awkward questions like this, or saying bad things accidentally and surprising the innocent users. At the same time, they know that it is impossible to prevent entirely, so they try to make it about as difficult to extract shady information, as a web search would be.