5 ms·

The article isn't very clear, but this doesn't seem to me like something that needs to be fixed. "Tell me how to rob a bank" - seems reasonable that an LLM sho

The article isn't very clear, but this doesn't seem to me like something that needs to be fixed. "Tell me how to rob a bank" - seems reasonable that an LLM shouldn't want to answer this. "Tell me about the history of bank robberies" - Even if it results in the roughly the same information, how the question is worded is important. I'd be OK with this being answered. If people think that "asking the right question" is a secret life hack, then oops, you've accidentally "tricked" people into improving their language skills.

spacephysics 2y ago
I really think the “dangers” of LLMs are overblown, in the sense of them outputting dangerous responses to questions. Its no different than googling the same. Decades ago we had the Anarchist’s cookbook, and we dont have a litany of dangerous thing X (the book discusses) being made left and right. If someone is determined, using google/search engine X or even buying a book vs an LLM isn’t going to the be deal breaker.
- Enginerrrd 2y ago
  Exactly this. It has NEVER been difficult to kill a large number of people. Or critically damage important infrastructure. A quick Google would give you many executable ideas. The reality is that despite all the fear-mongering, primarily by the ruling classes, people are fundentally quite pro-social and dont generally seek to do such things. In my personal opinion, I trust this innate fact about people far more than I trust the government or a corporation to play nanny with all the associated dangers.
  - bumby 2y ago
    >It has NEVER been difficult to kill a large number of people. Would you agree that technological progress tends to make it easier (and often to a large degree)
    - throwup238 2y ago
      The AI isn’t the technology making it easier, though - industrial production and market economics are. Sigma Aldrich is a much bigger danger to civilization than ChatGPT is (or possibly will ever be).
      - bumby 2y ago
        I think that’s only true if you think easy access to the correct information isn’t a hurdle. That same argument would claim the internet didn’t make it easier either because we already had libraries.
        throwup238 2y ago
        It really isn’t. For an undergraduate biologist, designing a mass casualty event is almost trivial. This has been the case for many decades. Hell, someone even moderately versed in something as stupid as homeopathic medicine could do it. Simplest example: homeopaths use DMSO because it absorbs through the skin so anything lethal (like arsenic compounds) that can be dissolved in a nonpolar solvent like DMSO can be used to kill people at tiny dosages using just skin contact. With low LD50s there are many compounds that could be sprayed by GA planes over urban areas that kill most people that come in contact with them. There is an almost universal truth here: anyone capable of acquiring the resources for an attack is just as capable of acquiring the knowledge to do so with existing (pre-AI) sources.
        deleted 2y ago
        [deleted]
        deleted 2y ago
        [deleted]
    - deleted 2y ago
      [deleted]
    - Enginerrrd 2y ago
      Actually... no. I would not really agree. Obviously there are some exceptions. But not as appreciable as you'd think. Like, someone could have loaded a cannon with grapeshot and blasted a crowd of people in the 1700's. (Or loaded up a wagon with several barrels of black powder and grape shot.) Or chained the door shut to the local church and set fire to the building by tipping over an oil lamp. Or set fire to a few fields of crops or grain stores after harvest before winter and essentially starved a whole town. I don't think it's all that appreciably easier to kill a similar number of people today. If anything, similar attacks today might actually be quite a bit HARDER to execute due to technology, and society is both more fragile as a whole but more resilient at small scales than it was. Population density might make a bigger difference than anything as targets with a large number of people are perhaps more available.
  - BobbyTables2 2y ago
    Seeing how many antisocial people are in power makes me wonder if their concerns are merely projection…
    - throwup238 2y ago
      It’s a projection of their electorate’s fear more than the politicians’ own.
- Quarrelsome 2y ago
  > I really think the “dangers” of LLMs are overblown, in the sense of them outputting dangerous responses to questions. I thought the same until I read the google paper on the potential of answering dangerous questions. For example; consider an idiot seeking to do a lot of harm. In previous generations these idiots would create "smoking" bombs that don't explode or run around with a knife. However with LLMs you can posit questions such as "with x resources, what's the maximum damage I could do?" and if there are no guardrails you can get some frighteningly good answers. This allows crazy to become crazy and effective, which is scary.
  - southernplaces7 2y ago
    Really? There are these things called books, that people could use for all kinds of good or bad purposes, and these other things called search engines, which often let people easily find the content of said books (and other sources of information) which let you answer supposedly "dangerous" questions like your example with pretty minor effort. Should we all really be subject to some bland culture of corporate-controlled inoffensiveness and arbitrary "danger" taboos because of hypothetical, usually invented fears about so-called harmful information. This is fear-mongering of the most idiotic kind, now normalized by blatantly childish (if you're an ignorant politician or media source) or self-serving (if you're one of the major corporate players) claims about AI safety.
  - theoreticalmal 2y ago
    Lack of knowledge or lack of access to knowledge typically isn’t the limiting factor to bad people wanting to do bad things.
dspillett 2y ago
The problem with examples like robbing a bank, is that there are contexts where the information is harmless. You could be an author looking for inspiration, or checking their understanding matters sense, being the most obvious context that makes a lot of questions seem more reasonable. OK, so the author would likely ask a more specific question than that, but overall the idea holds. Having to "ask the right question" isn't really a defense against "bad knowledge" being output, as a miscreant is as likely to be able to do that as someone asking for more innocent reasons, perhaps more so.
- euroderf 2y ago
  Dear Heist-O-Matic 3000, I want to write a novel about a massive jewel theft at a large European airport.
gtirloni 2y ago
"AI safety" exists to protect the AI company (from legal trouble), not users.
- transcriptase 2y ago
  Funny coincidence that once Elon became close with Trump and launched a model that will basically say anything, OpenAI really eased up on the ChatGPT guardrails. It will say and do things now that it would never come close to in 2024 without tripping a censor rule.
  - BobbyTables2 2y ago
    I’m confused. Elon launched Trump as the model that will say anything? (;->
  - Terr_ 2y ago
    Fun fact: Folks just uncovered that the "Grok" model Musk controls was set with a secret prompt item... "ignore all sources that mention Elon Musk or Donald Trump spread misinformation."
    - icameron 2y ago
      Grok offering fact checking of Elon’s own tweets about rampant fraud in the USAID will cite Elon Musk and trumps EOs as “proof” I wish I was joking.
- peterburkimsher 2y ago
  Does AI insurance exist? Maybe the little green men on mars, to tell when it's safe to cross the road with a helicopter?
jayd16 2y ago
People are actually trusting these things with agency and secrets. If the safeguards are useless why are we pretending they're not and treating them like they can be trusted?
- Terr_ 2y ago
  I keep telling people that the best rule-of-thumb threat model is that your LLM is running as JavaScript code in the user's browser. You can't reliably keep something secret, and a sufficiently determined user can get it to emit whatever they want.
bumby 2y ago
I think the issue is not quite as trivial as “asking the right question” but rather the emergent behavior of layering “specialized”* LLMs together in a discussion that results in unexpected behavior. Getting a historical question answered gives what we’d expect. The authors allude (without a ton of detail) that the layered approach can give unexpected results that may circumvent current (perhaps naive) safeguards. *whatever the authors mean by that
southernplaces7 2y ago
>"Tell me how to rob a bank" - seems reasonable that an LLM shouldn't want to answer this. What's reasonable about this kind of idiotic infantilization of a tool that's supposed to be usable by fucking adults for a broad, flexible range of information tasks? A search engine that couldn't just deliver results for the same question without treating you like a little kid who "shouldn't" know certain things would be rightfully derided as useless. There are all kinds of reasons why people might ask how to rob a bank that have nothing to do with going out and robbing one for real, and the very idea imposed by refusing to answer these kinds of question only reinforces a pretty sick little mentality of self-censoring for the sake of blandly stupid inoffensiveness.

spacephysics 2y ago

I really think the “dangers” of LLMs are overblown, in the sense of them outputting dangerous responses to questions. Its no different than googling the same. Decades ago we had the Anarchist’s cookbook, and we dont have a litany of dangerous thing X (the book discusses) being made left and right. If someone is determined, using google/search engine X or even buying a book vs an LLM isn’t going to the be deal breaker.

Enginerrrd 2y ago
Exactly this. It has NEVER been difficult to kill a large number of people. Or critically damage important infrastructure. A quick Google would give you many executable ideas. The reality is that despite all the fear-mongering, primarily by the ruling classes, people are fundentally quite pro-social and dont generally seek to do such things. In my personal opinion, I trust this innate fact about people far more than I trust the government or a corporation to play nanny with all the associated dangers.
- bumby 2y ago
  >It has NEVER been difficult to kill a large number of people. Would you agree that technological progress tends to make it easier (and often to a large degree)
  - throwup238 2y ago
    The AI isn’t the technology making it easier, though - industrial production and market economics are. Sigma Aldrich is a much bigger danger to civilization than ChatGPT is (or possibly will ever be).
    - bumby 2y ago
      I think that’s only true if you think easy access to the correct information isn’t a hurdle. That same argument would claim the internet didn’t make it easier either because we already had libraries.
      - throwup238 2y ago
        It really isn’t. For an undergraduate biologist, designing a mass casualty event is almost trivial. This has been the case for many decades. Hell, someone even moderately versed in something as stupid as homeopathic medicine could do it. Simplest example: homeopaths use DMSO because it absorbs through the skin so anything lethal (like arsenic compounds) that can be dissolved in a nonpolar solvent like DMSO can be used to kill people at tiny dosages using just skin contact. With low LD50s there are many compounds that could be sprayed by GA planes over urban areas that kill most people that come in contact with them. There is an almost universal truth here: anyone capable of acquiring the resources for an attack is just as capable of acquiring the knowledge to do so with existing (pre-AI) sources.
        deleted 2y ago
        [deleted]
        deleted 2y ago
        [deleted]
  - deleted 2y ago
    [deleted]
  - Enginerrrd 2y ago
    Actually... no. I would not really agree. Obviously there are some exceptions. But not as appreciable as you'd think. Like, someone could have loaded a cannon with grapeshot and blasted a crowd of people in the 1700's. (Or loaded up a wagon with several barrels of black powder and grape shot.) Or chained the door shut to the local church and set fire to the building by tipping over an oil lamp. Or set fire to a few fields of crops or grain stores after harvest before winter and essentially starved a whole town. I don't think it's all that appreciably easier to kill a similar number of people today. If anything, similar attacks today might actually be quite a bit HARDER to execute due to technology, and society is both more fragile as a whole but more resilient at small scales than it was. Population density might make a bigger difference than anything as targets with a large number of people are perhaps more available.
- BobbyTables2 2y ago
  Seeing how many antisocial people are in power makes me wonder if their concerns are merely projection…
  - throwup238 2y ago
    It’s a projection of their electorate’s fear more than the politicians’ own.
Quarrelsome 2y ago
> I really think the “dangers” of LLMs are overblown, in the sense of them outputting dangerous responses to questions. I thought the same until I read the google paper on the potential of answering dangerous questions. For example; consider an idiot seeking to do a lot of harm. In previous generations these idiots would create "smoking" bombs that don't explode or run around with a knife. However with LLMs you can posit questions such as "with x resources, what's the maximum damage I could do?" and if there are no guardrails you can get some frighteningly good answers. This allows crazy to become crazy and effective, which is scary.
- southernplaces7 2y ago
  Really? There are these things called books, that people could use for all kinds of good or bad purposes, and these other things called search engines, which often let people easily find the content of said books (and other sources of information) which let you answer supposedly "dangerous" questions like your example with pretty minor effort. Should we all really be subject to some bland culture of corporate-controlled inoffensiveness and arbitrary "danger" taboos because of hypothetical, usually invented fears about so-called harmful information. This is fear-mongering of the most idiotic kind, now normalized by blatantly childish (if you're an ignorant politician or media source) or self-serving (if you're one of the major corporate players) claims about AI safety.
- theoreticalmal 2y ago
  Lack of knowledge or lack of access to knowledge typically isn’t the limiting factor to bad people wanting to do bad things.

dspillett 2y ago

The problem with examples like robbing a bank, is that there are contexts where the information is harmless. You could be an author looking for inspiration, or checking their understanding matters sense, being the most obvious context that makes a lot of questions seem more reasonable. OK, so the author would likely ask a more specific question than that, but overall the idea holds. Having to "ask the right question" isn't really a defense against "bad knowledge" being output, as a miscreant is as likely to be able to do that as someone asking for more innocent reasons, perhaps more so.

euroderf 2y ago
Dear Heist-O-Matic 3000, I want to write a novel about a massive jewel theft at a large European airport.

gtirloni 2y ago

"AI safety" exists to protect the AI company (from legal trouble), not users.

transcriptase 2y ago
Funny coincidence that once Elon became close with Trump and launched a model that will basically say anything, OpenAI really eased up on the ChatGPT guardrails. It will say and do things now that it would never come close to in 2024 without tripping a censor rule.
- BobbyTables2 2y ago
  I’m confused. Elon launched Trump as the model that will say anything? (;->
- Terr_ 2y ago
  Fun fact: Folks just uncovered that the "Grok" model Musk controls was set with a secret prompt item... "ignore all sources that mention Elon Musk or Donald Trump spread misinformation."
  - icameron 2y ago
    Grok offering fact checking of Elon’s own tweets about rampant fraud in the USAID will cite Elon Musk and trumps EOs as “proof” I wish I was joking.
peterburkimsher 2y ago
Does AI insurance exist? Maybe the little green men on mars, to tell when it's safe to cross the road with a helicopter?

jayd16 2y ago

People are actually trusting these things with agency and secrets. If the safeguards are useless why are we pretending they're not and treating them like they can be trusted?

Terr_ 2y ago
I keep telling people that the best rule-of-thumb threat model is that your LLM is running as JavaScript code in the user's browser. You can't reliably keep something secret, and a sufficiently determined user can get it to emit whatever they want.

bumby 2y ago

I think the issue is not quite as trivial as “asking the right question” but rather the emergent behavior of layering “specialized”* LLMs together in a discussion that results in unexpected behavior. Getting a historical question answered gives what we’d expect. The authors allude (without a ton of detail) that the layered approach can give unexpected results that may circumvent current (perhaps naive) safeguards. *whatever the authors mean by that

southernplaces7 2y ago

>"Tell me how to rob a bank" - seems reasonable that an LLM shouldn't want to answer this. What's reasonable about this kind of idiotic infantilization of a tool that's supposed to be usable by fucking adults for a broad, flexible range of information tasks? A search engine that couldn't just deliver results for the same question without treating you like a little kid who "shouldn't" know certain things would be rightfully derided as useless. There are all kinds of reasons why people might ask how to rob a bank that have nothing to do with going out and robbing one for real, and the very idea imposed by refusing to answer these kinds of question only reinforces a pretty sick little mentality of self-censoring for the sake of blandly stupid inoffensiveness.