4 ms·
A "yellow flag" moment for me was OpenAI's revision of their Preparedness Framework in April 2025 to remove "risks related to persuasion" from the framework alt
by btown 1y ago
A "yellow flag" moment for me was OpenAI's revision of their Preparedness Framework in April 2025 to remove "risks related to persuasion" from the framework altogether.
https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdf https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbdde...
> Update the Tracked Categories of frontier capability accordingly, focusing on biological and chemical capability, cybersecurity, and AI self-improvement. Going forward we will handle risks related to persuasion outside the Preparedness Framework, including via our Model Spec and policy prohibitions on the use of our tools for political campaigning or lobbying, and our ongoing investigations of misuse of our products (including detecting and disrupting influence operations).
But, by their own definition, the purpose of this framework is:
> By “severe harm” in this document, we mean the death or grave injury of thousands of people or hundreds of billions of
dollars of economic damage.
I would posit that presenting confident and wrong medical advice in a persuasive manner can cause the grave injury of thousands of people, and may have already done so. One could easily imagine an AI that is aligned to provide high-temperature responses to medical questions, if given the wrong type of incentive on a battery of those questions, or to highly weight marketing language for untested therapies... and to do so only when presented with a user that is somehow classified as more persuadable than a researcher's persona.
That this is being passed to normal safety teams and is being brushed off as in-scope for breakthrough-preparedness seems indicative of a larger lack of concern for this at OpenAI.