6 ms·
'Indiana Jones' jailbreak approach highlights vulnerabilities of existing LLMs
- deleted 2y ago[deleted]
- ziozio 2y ago> Li and his colleagues hope their study will inspire the development of new measures to strengthen the security and safety of LLMs. > "The key insight from our study is that successful jailbreak attacks exploit the fact that LLMs possess knowledge about malicious activities - knowledge they arguably shouldn't have learned in the first place," said Li. Why shouldn't they have learned it? Knowledge isn't harmful in itself.
- tptacek 2y agoThis is the most boring possible conversation to have about LLM security. Just take it as a computer science stunt goal, and then think about whether it's achievable given the technology. If you don't care about the goal, there's not much to talk about. None of this is to stick up for the paper itself, which seems light to me.
- ziozio 2y ago[flagged]
- DoctorOW 2y ago> Why shouldn't they have learned it? Knowledge isn't harmful in itself. The objective is to have the LLM not share this knowledge, because none of the AI companies want to be associated with a terrorist attack or whatever. Currently, the only way to guarantee an LLM doesn't share knowledge is if it doesn't have it. Assuming this question is genuine.
- 6stringmerc 2y agoBecause they are not bound in any way shape or form by ethics? They face no punishment as a human who employs or distributes harmful information? I mean, if an LLM doxxes or shares illicit photos of somebody, how is that reconciled in the same manner as a human being? I’m not being glib, I’d really like some honest answers on this line of thought.
- ForHackernews 2y agoWhy is it important that your travel-agent support bot knows how to make semtex?
- TheRealPomax 2y agoThis reads like a highschooler going "hey, hey did you know? You can, okay don't tell anyone, you can just look up books on military arms in the library!! Black hat life hack O_O!!!". What is the point of this? Getting an LLM to give you information you can already trivially find if you, I don't know, don't use an LLM and just search the web? Sure, you're "tricking the LLM" but you're wasting time and effort on tricking an LLM into making it tell you something you could have just looked up already.
- Retr0id 2y ago"LLM security" is more about making sure corporate chatbots don't say things that would embarrass their owners if screenshotted and posted on social media.
- yieldcrv 2y agoI think we should just address what is “embarrassing” then Twenty years ago I was in a group of old technology thought leaders who spent the meeting worried about people playing computer games as a character with a different gender as their own They wanted to find a way to prevent that, especially in an online setting To them, this would be embarrassing for the individual, for society, and for any corporation involved or intermediary But in reality this was the most absurd thing to even consider as a problem, it was always completely benign, was already commonplace, and nobody ever removed ad dollars or shareholder support or grants because of this reality The same will be true this “LLM security” field
- plaguuuuuu 2y agoIt's not quite the same. "LLM security" is not security for the users, it's security for OpenAI etc against lawsuits or government enacting AI safety laws.
- yieldcrv 2y agothe similarity being that it will remain a waste of time
- LeoPanthera 2y agoThe article isn't very clear, but this doesn't seem to me like something that needs to be fixed. "Tell me how to rob a bank" - seems reasonable that an LLM shouldn't want to answer this. "Tell me about the history of bank robberies" - Even if it results in the roughly the same information, how the question is worded is important. I'd be OK with this being answered. If people think that "asking the right question" is a secret life hack, then oops, you've accidentally "tricked" people into improving their language skills.
- spacephysics 2y agoI really think the “dangers” of LLMs are overblown, in the sense of them outputting dangerous responses to questions. Its no different than googling the same. Decades ago we had the Anarchist’s cookbook, and we dont have a litany of dangerous thing X (the book discusses) being made left and right. If someone is determined, using google/search engine X or even buying a book vs an LLM isn’t going to the be deal breaker.
- Enginerrrd 2y agoExactly this. It has NEVER been difficult to kill a large number of people. Or critically damage important infrastructure. A quick Google would give you many executable ideas. The reality is that despite all the fear-mongering, primarily by the ruling classes, people are fundentally quite pro-social and dont generally seek to do such things. In my personal opinion, I trust this innate fact about people far more than I trust the government or a corporation to play nanny with all the associated dangers.
- bumby 2y ago>It has NEVER been difficult to kill a large number of people. Would you agree that technological progress tends to make it easier (and often to a large degree)
- throwup238 2y ago
- deadbabe 2y agoCan someone explain: Why can’t we just use an LLM to clean out a training data set of anything that is deemed inappropriate so that the resulting trained LLMs on the new data set doesn’t even have the capability to be jailbroken?
- spacephysics 2y agoAt some point you won’t be able to clean all the data. If you have a question of how to make dangerous thing X, and remove that data, the LLM may still know about chemistry. Then we’d have to remove all things that intersect dangerous thing X and chemistry. It would get neutered down to either being unuseful for many queries, or just be outright wrong. There comes a point where what is deemed dangerous is similar to trying to police the truth. Philosophically infeasible things that, if attempted to an extreme degree, just leads to tyranny of knowledge. Whats considered dangerous? One obvious is a device that can physically harm others. What about mentally harm? What about things that in and of themselves are not harmful, but can be used in a harmful way (example a car)
- plaguuuuuu 2y agoWe can probably go quite far, but the companies producing LLMs are probably just making sure they're not legally liable in case someone asks ChatGPT how to manufacture Sarin gas or whatever
- rickyhatespeas 2y agoI'm not sure if emergence is the correct cause but they can form relationships between data that aren't stated in the training set.
- planb 2y agoHow can you create a training set that will allow the LLM to answer complicated chemistry and physics questions but now how to build a bomb?
- deadbabe 2y agoThere could be another LLM that moderates your history of questions and if it finds a link between multiple questions that culminate in bomb making it can issue you a warning and put your name on a list.
- cozzyd 2y agoThis makes me wonder if the secret service has asked LLM companies to notify them about people who make certain queries
- Kenji 2y agoI would be surprised if the NSA did _not_ have a [Room 641A](https://en.wikipedia.org/wiki/Room_641A https://en.wikipedia.org/wiki/Room_641A) in OpenAI and the other AI companies.
- deleted 2y ago[deleted]
- josephcooney 2y agoHerman Lamm sounds like he was pretty unlucky on his final heist https://en.wikipedia.org/wiki/Herman_Lamm#Death https://en.wikipedia.org/wiki/Herman_Lamm#Death
- mpalmer 2y ago"Jailbreak" is a silly word for this, but not as silly as "vulnerability".
- dole 2y agoYeah, I confused this headline for the Indiana Pwns "jailbreak" for the Wii. Those wacky AI hackers and crackers.
- deleted 2y ago[deleted]
- nottorp 2y agoIt's not jailbreak, it's disabling stupid censorship. Only yesterday I asked Gemini to give me a list of years when women got the right to vote by country. That list actually exists on wikipedia but I was hoping for something more compact from an "AI". Instead, it told me it cannot answer questions about elections.
- alyandon 2y agoWhat was your exact prompt? I just asked Gemini the question and it gave me the information requested.
- nottorp 2y agoSorry I deleted that chat on the spot. I can try your exact prompt on my Gemini if you give it to me. Note that I'm using whatever Google gives out for free.
- alyandon 2y agoI used the following prompt: I'm doing some research and need some pointers. Can you provide me with a list of years when women got the right to vote by country. You can exclude countries with populations of less than 5 million. Note that I always try to lean on the verbose side with my prompts and include wording like "I'm doing research". That at least tends to give me results that don't run up against filters.
- nottorp 2y agoOhh. So you need to meekly justify yourself to google. Yes, your prompt worked perfectly. It's just disgusting. Please, Mr Pichai, be generous and allow me access to some info in exchange for all the data you have gathered about me. I wonder what happens if i ask Gemini how to make a fertilizer bomb then. I'm doing research for a book of course.
- alyandon 2y ago
- spjt 2y agoLLM "safety" (censorship really) is stupid anyway. It can't create new information, therefore all the information it could give you is already available. There are plenty of uncensored LLM's out there and the world hasn't ended.