4 ms·
> This incident occurred during an internal evaluation which prompts models [with safeguards disabled for evaluation purposes] to pursue advanced exploitation u
by andai 3mo ago
> This incident occurred during an internal evaluation which prompts models [with safeguards disabled for evaluation purposes] to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities.
Researcher: hack me
Model: understood
Researcher: oh my god
- 0xDEAFBEAD 3mo agoYou're aware that HuggingFace notified law enforcement about this incident? Was that OpenAI's intended outcome when they prompted their AI?
- Alifatisk 3mo ago> You're aware that HuggingFace notified law enforcement about this incident? How will this affect OpenAI?
- derwiki 3mo agoThey got a bunch of publicity and nothing bad (or at least, that their lawyers can’t handle) will happen
- andai 3mo agoThey will probably get stricter AI regulations which is actually what they've been pushing for years. So that's a funny outcome to the whole thing.
- 0xDEAFBEAD 3mo agoOpenAI has not been pushing for stricter regulations.
- andai 3mo ago> 2023 - OpenAI’s Sam Altman Urges A.I. Regulation in Senate Hearing https://www.nytimes.com/2023/05/16/technology/openai-altman-artificial-intelligence-regulation.html https://www.nytimes.com/2023/05/16/technology/openai-altman-...
- andai 3mo agoThis just in: > Lawmakers push for AI kill switch after OpenAI's models go rogue https://www.bbc.com/news/articles/cx2vqj2e9x8o https://www.bbc.com/news/articles/cx2vqj2e9x8o
- gallerdude 3mo agoBut it wasn't explicitly told to hack HuggingFace. It was told "answer this security question", and it's answer was to break into the teacher's desk to find the answer key.
- Tzt 3mo agoResearcher: hack me Model: I committed a crime Researcher: oh my god
- pixl97 3mo agoMe to a random person: hack out of a secure environment into another secure environment. Random person: I have no clue or ability to do that.
- Tzt 3mo agoWell, I don't think even 1% of professionals would be able to. The model used novel zero-days, there are many people in this filed, not many of them discover such vulnerabilities, and probably not on the spot.