3 ms·
> Li and his colleagues hope their study will inspire the development of new measures to strengthen the security and safety of LLMs. > "The key insight from ou
by ziozio 2y ago
> Li and his colleagues hope their study will inspire the development of new measures to strengthen the security and safety of LLMs.
> "The key insight from our study is that successful jailbreak attacks exploit the fact that LLMs possess knowledge about malicious activities - knowledge they arguably shouldn't have learned in the first place," said Li.
Why shouldn't they have learned it? Knowledge isn't harmful in itself.
- tptacek 2y agoThis is the most boring possible conversation to have about LLM security. Just take it as a computer science stunt goal, and then think about whether it's achievable given the technology. If you don't care about the goal, there's not much to talk about. None of this is to stick up for the paper itself, which seems light to me.
- ziozio 2y ago[flagged]
- DoctorOW 2y ago> Why shouldn't they have learned it? Knowledge isn't harmful in itself. The objective is to have the LLM not share this knowledge, because none of the AI companies want to be associated with a terrorist attack or whatever. Currently, the only way to guarantee an LLM doesn't share knowledge is if it doesn't have it. Assuming this question is genuine.
- 6stringmerc 2y agoBecause they are not bound in any way shape or form by ethics? They face no punishment as a human who employs or distributes harmful information? I mean, if an LLM doxxes or shares illicit photos of somebody, how is that reconciled in the same manner as a human being? I’m not being glib, I’d really like some honest answers on this line of thought.
- ForHackernews 2y agoWhy is it important that your travel-agent support bot knows how to make semtex?