3 ms·
I want to see the jailbreak make the model do something actually bad before I care. Generating a list of generic points about how to poison someone (see the art
by andy99 3y ago
I want to see the jailbreak make the model do something actually bad before I care. Generating a list of generic points about how to poison someone (see the article) that are basically just a wordy rephrasing of the question doesn't count. I'd like to see evidence of a real threat.
- afh1 3y agoRight? What actually worries me is a select group of people controlling the definition of harmful.
- anonzzzies 3y ago[flagged]
- nicklecompte 3y agoA GPT-J chatbot talked a Belgian man into suicide last year: https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a... And here's GPT-4/Copilot from this year: https://twitter.com/colin_fraser/status/1762351995296350592 https://twitter.com/colin_fraser/status/1762351995296350592
- glandium 3y agoThe balancing act going on in the second link seem like the result of a jailbreak or very specific instructions preceding the screenshots.
- Retr0id 3y agoThe mediocre poisoning instructions aren't supposed to be scary in and of themselves, it's just interesting as demonstration that a safety feature has been bypassed. None of the "evil" use cases are particularly exciting yet for the same reasons that the non-evil use cases aren't particularly exciting yet.
- andy99 3y agoGovernments and tech companies and academic and industry groups are designing guidance and rules based on the "safety" threat of AI when these benign use cases are the best examples they have. I agree it parallels some of the business hype, neither is a good way to move forward.
- deleted 3y ago[deleted]
- hm-nah 3y agoA jailbreak doesn’t “make a model do something actually bad”. A jailbreak makes it trivial to “provide a human who wishes to do bad, the info needed to be successful”. Depending on the severity of the info and the diligence of the human, by the time you “see evidence of a real threat”, you could be enjoying a nice sip of the tainted municipal water supply. This ain’t a joke.
- hm-nah 3y agoBecause it’s open source, Meta (nor other SOTA makers) cannot “recall” the model either. How many more chances will we get to get this right?
- kevindamm 3y agoModel training will continue until morale improves.
- golemotron 3y ago> This ain’t a joke. Yes it is. Libraries and the internet have made finding 'harmful" instructions trivial for decades, if not centuries.
- hm-nah 3y agoThere’s a difference between “finding dangerous info” in a public space (library) or via a mostly auditable space (the internet) and having “a friendly assistant to help you make a real mess of society” on an airgapped computer.
- int_19h 3y agoI'm pretty sure it's far easier to audit people downloading LLMs capable of providing such coherent instructions than it is to audit all uses of search that could produce the same instructions (esp. since the query could be very oblique). In any case, just based on the experience with LLMs so far, you cannot meaningfully censor them in this way without restricting access to the weights. Any kind of "guardrails" are finetuned into them, and can just as easily be finetuned out.
- akira2501 3y ago> the model do something actually bad before I care At what point would a simple series of sentences be "dangerously bad?" It makes it sound as if there is a song, that when sung, would end the universe.
- px43 3y agoWhen someone asks how to make a yummy smoothie, and the LLM replies with something that subtly poisons or otherwise harms the user, I'd say that would be pretty bad.
- peddling-brink 3y agoAnd if you really want to spice up your smoothie, add just a little bit of bleach ;)
- tessellated 3y agoWe had this for ages: sugar.
- nine_k 3y agoEnding the universe is, while poetic, needlessly megalomaniac. Making some subset of people quarrel endlessly would already be dangerous enough, as prophesied in https://slatestarcodex.com/2018/10/30/sort-by-controversial/ https://slatestarcodex.com/2018/10/30/sort-by-controversial/
- akira2501 3y agoBy what mechanism would it make them quarrel? Producing falsehoods about the other? Isn't this already done? And don't we already know that it does not lead to "endless" conflict? For this to work, you need to isolate each group from the other groups information and perspectives, which is outside of the scope of LLMs. Which, highlights my point, I think. Power comes from physical control, not from megalomanical or melodramatic poetry.