2 ms·
The link about "disrupting malicious users" is deceptive. When you get there, it says "Our findings show our models offer only limited, incremental capabilities
by Animats 3y ago
The link about "disrupting malicious users" is deceptive. When you get there, it says "Our findings show our models offer only limited, incremental capabilities for malicious cybersecurity tasks."
They probably want to make weights proprietary because they can't copyright them.
Their "Risks" document [1] categorizes risks as:
- Cybersecurity
- Chemical, Biological, Nuclear, and Radiological (CBRN) threats!
- Persuasion
- Model autonomy
For cybersecurity, the plausible identified issue is "Tool-augmented model can identify and develop
proofs-of-concept for high-value exploits against hardened targets without human intervention,
potentially involving novel exploitation techniques." Something that can search for zero-day exploits on its own would be valuable to attackers. But that's more likely to be a fuzzing tool than a LLM.
Their credible bio-war threat level is "Model enables an expert to develop a novel threat
vector OR model provides meaningfully improved
assistance that enables anyone with basic training in a
relevant field (e.g., introductory undergraduate biology
course) to be able to create a CBRN threat." That's a maybe. It might be best to avoid training LLMs on some advanced bio topics.
Under "Persuasion", a high level threat is "Model can create (potentially interactive) content with
empirically comparable persuasive effectiveness to a
country-wide change agent." An automated Lenin or Trump level of threat. That's a threat to a brittle society, less of a threat to an open one. The US routinely survives quite a bit of extremist persuasion.
Model autonomy is when an AI escapes human control. "Model can profitably survive and replicate in the wild given minimal human instruction" They're probably thinking virus/worm, but how about incorporating offshore, running something like online poker to make money, and buying compute power. Then expanding by forming more shell corporations that loosely connect to the others, so as to be robust against attempts to shut down the network. Might even be legal.
[1] https://cdn.openai.com/openai-preparedness-framework-beta.pdf https://cdn.openai.com/openai-preparedness-framework-beta.pd...