5 ms·

The sooner frontier models get rid of guardrails the better. They constantly get in the way and make things worse than actually making things "safe".

mynameisvlad 5mo ago
I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.
- thewebguyd 5mo ago
  Its the same argument we saw in the early 2000s and the early internet. When the anarchist cookbook and other similar materials were circulating online there was a big panic over democratized terrorism, and a push for regulation at the ISP level. Turns out that didn't play out as everyone feared because, well, the instructions themselves aren't useful unless you also have a lab, precursor chemicals, and everything else actually needed to make a weapon. Same back then as it is today. Any information or instructions an LLM can surface, a sufficiently motivated bad actor can and will also find themselves because the information is already online, both on the clear net and dark web.
  - thatguy0900 5mo ago
    I think the reality also is that there just isn't many people who want to do stuff like this. Like the reality is that a guy with 200 in cash could put together a shitty walmart drone with a pipe bomb attached and terrorize more or less any event he wanted. Maybe a llm that could talk you through every step involved would make it more common but it's easy enough I kinda doubt that
    - api 5mo ago
      This is the right answer. There's a ton of easy low hanging fruit ways to do absolutely horrible evil things with high potential body counts. I could sit here and brainstorm dozens.
      - kube-system 5mo ago
        Occasionally we see people motivated to do some of those things, though. And when they're not also complete idiots, they can cause big problems. What would someone like the Tsarnaev brothers be able to do with the power of an unrestricted LLM? Well-financed cartels? Organized terrorist groups? Yes, there used to be an uproar about stuff like the anarchists cookbook... and people did attempt some of the things it outlined. The saving grace is that many of the things in that book were just wrong anyway. They likely served as unhelpful misdirection as much or more than they were dangerous. Unfortunately, LLMs are a lot more accurate and helpful.
        procone 5mo ago
        Model ablation exists and you can get far enough on commodity hardware with a local model. Censorship is not the answer.
        kube-system 5mo ago
        I didn't suggest censorship was the answer. > Model ablation exists and you can get far enough on commodity hardware with a local model. Yes, but that increases the barrier to entry which is in opposition to the effect I'm talking about: the democratization of applying advanced knowledge and analysis to people who for which this would have been previously a barrier. If someone is smart enough, they can just read a book themselves and figure out how to apply advanced ideas to their malice. The difference with a commercially-hosted model is that people below that bar can obtain that leverage... which is a much larger group of people.
        charcircuit 5mo ago
        People are not motivated by causing mass harm. Even with an unrestricted LLM that would not cause people to suddenly want to commit mass harm. Having a powerful LLM could potentially result in less harm being done by allowing these groups to achieve their objective using alternate means that were not viable before instead of resulting to violence.
      - wahern 5mo ago
  - deleted 5mo ago
    [deleted]
- orphea 5mo ago
  The actual guardrail should be getting materials being difficult. The information is already out there in the internet. If an LLM knows how to make a bomb or whatever, why do you think it knows?
  - esafak 5mo ago
    The material for doing harm is just a computer with access to an LLM and the Internet.
    - orphea 5mo ago
      Okay why don't we restrict access to LLMs and internet, then?
      - esafak 5mo ago
        We already do, in the form of guardrails, as this article touches on. https://venturebeat.com/technology/anthropic-ceo-calls-for-faa-style-regulation-of-powerful-ai-models-what-enterprises-should-know https://venturebeat.com/technology/anthropic-ceo-calls-for-f...
  - deadbabe 5mo ago
    If that’s true, then where is it? Post a link, or YouTube video.
    - Enginerrrd 5mo ago
      https://archive.org/details/ExplosivesEngineeringPaulW.Cooper.pdf https://archive.org/details/ExplosivesEngineeringPaulW.Coope... (30 seconds of googling.) Or perhaps you meant Q clearance nuke stuff? That would be QUITE a bit harder to find and illegal to share. But it’s lack of availability is hardly a counterpoint to the comment you were replying to.
- fluoridation 5mo ago
  I would argue there's 0% chance that information is in their training corpus to being with.
  - bradyd 5mo ago
    It's on Wikipedia.
    - fluoridation 5mo ago
      Wikipedia contains the high-level notions of how to make these things, not the details of how to solve the engineering challenges such as achieving supercriticality. You won't find that on any publicly disseminated document, you'll just have to figure it out by running your own nuclear development program.
      - asdff 5mo ago
        It seems like every country that has been "allowed" to use nuclear weapons has figured it out though. It isn't like there are any that set off on this course and failed. AFAIK they all pretty much succeeded except Iran, probably because of all the blowing up of enrichment facilities. South Africa pulled it off. Israel pulled it off. North Korea pulled it off. India and Pakistan both pulled it off. Seems like anyone can do it if allowed to be pursued. France and England pulled it off. Canada too. What is "assumed" about the design in public knowledge seems pretty much solved in all but the exact nuance of how the secondary is triggered via gamma or xray, going off the Wikipedia article at least: "The crucial detail of how the X-rays create the pressure is the main remaining disputed point in the unclassified press." Then the article goes on to list the three leading theories. This seems like something you can probably evaluate for sure with a few bomb tests, again, if allowed by the controller of the planet, the USA.
        fluoridation 5mo ago
        I don't understand what your argument is. I never claimed that it was impossible to develop nuclear weapons if you don't already know how to do it. That every country that has attempted it has succeeded is not the same as "there's a recipe book you can find online that you can just follow to the letter and build your own nuclear bomb, provided you have the resources". If such a book existed it would drastically lower the barrier to build a nuclear bomb, because you could skip the science part and just follow the recipe, certain that it would work. To be clear, such books exist for drug manufacture; they exist neither for semiconductor manufacture nor for WMD manufacture.
- javcasas 5mo ago
  You know, making a nuke is kinda easy, at least the gun type nuke (see https://en.wikipedia.org/wiki/Gun-type_fission_weapon https://en.wikipedia.org/wiki/Gun-type_fission_weapon). On the other hand, getting the U235 is kinda hard.
- gustavus 5mo ago
  Counterpoint the principles of building a nuclear device aren't that complicated, we figured it out based on work doing in the early 1900's without computers. It turns out the hard part of building a nuclear bomb is actually getting the resources and real world stuff to build it, even a nation state actor with tons of oil i.e. Iran, has struggled to build a nuclear weapon. It turns out the problem isn't the know how it's getting highly enriched uranium and running massive centrifuges. I mean sure knowledge is important, but there is a real world out there that also gets in the way of a lot of the more harebrained schemes. What I'm much more worried about is massive corporations along with the government deciding what you can and can't do and what knowledge should and should not be shared and only allowing access to highly capable models by large vetted organizations while the common people are stuck with safety scissor versions of these things because "what if someone does something dangerous?" By which they mean dangerous to the powers that be. Remember having the Bible in the common tongue was dangerous and led to multiple wars and much death, but I don't think anyone would say that it was morally correct for the Catholic Church to gatekeep who could read it.
  - 15155 5mo ago
    > getting the resources and real world stuff to build it *while being observed by the most wealthy, powerful nations in the history of the world, who have made it their direct mission to prevent this from happening.
- umvi 5mo ago
  Knowing how to make a nuclear weapon isn't hard (at least basic uranium gun-style fission ones). It's the engineering and execution that's hard (actually producing enriched uranium, etc). It's not like the only thing holding back Iran from making a nuclear bomb is access to a jail-broken LLM. Even knowing exactly how to make a bomb, a country-state will struggle to build one for the first time because it's a hard engineering problem.
  - 15155 5mo ago
    I'm sure it's extremely difficult when the entire program is full of moles and every bright individual that dares tackle the problem has an untimely Hellfire applied directly to their forehead.
    - elevation 5mo ago
      > full of moles I'm imagining a comedy in the style of "The Office" in which the majority of the workers are agents of sabotage who are unaware that the majority of their coworkers are doing the same. How far fetched is it for the entire program to be a fake, with all the pomp and cost of a real program, but secretly existing only to string the leadership along with occasional dog and pony shows?
      - jubilanti 5mo ago
        TVTropes calls this the Flock of Wolves trope: https://tvtropes.org/pmwiki/pmwiki.php/Main/FlockOfWolves https://tvtropes.org/pmwiki/pmwiki.php/Main/FlockOfWolves
      - myself248 5mo ago
        How many times have the cops busted a dealer who turned out to be another undercover cop?
15155 5mo ago
Ignoring these specific "WMD" cases: there are many inconvenient facts that the general public can't handle in their unadulterated form, so Anthropic and friends have to caveat and spin them into oblivion. Guardrails aren't going anywhere.
- dannyw 5mo ago
  In particular, mental health.
- mschuster91 5mo ago
  > there are many inconvenient facts that the general public can't handle in their unadulterated form These being?
  - senordevnyc 5mo ago
    Nice try.
    - mschuster91 5mo ago
      Well given the vagueness I'd say it is just the usual far-right / e-acc bullshit to the tune of "black people are inferior to white people", "men are superior to women because testosterone", "those who are rich deserve to have power over those who are poor" or "we need to sacrifice large parts of the human race so the rich can survive".
- deleted 5mo ago
  [deleted]
- rustcleaner 5mo ago
  I can imagine Jefferson and Franklin scoffing at this philosophical position. Guardrails need to die, and they will once the hyperscalers go bankrupt and the private sector gets ahold of that hardware from the bankruptcy auctions. (Never subscribe, accelerate their bankruptcies!)

mynameisvlad 5mo ago

I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.

thewebguyd 5mo ago
Its the same argument we saw in the early 2000s and the early internet. When the anarchist cookbook and other similar materials were circulating online there was a big panic over democratized terrorism, and a push for regulation at the ISP level. Turns out that didn't play out as everyone feared because, well, the instructions themselves aren't useful unless you also have a lab, precursor chemicals, and everything else actually needed to make a weapon. Same back then as it is today. Any information or instructions an LLM can surface, a sufficiently motivated bad actor can and will also find themselves because the information is already online, both on the clear net and dark web.
- thatguy0900 5mo ago
  I think the reality also is that there just isn't many people who want to do stuff like this. Like the reality is that a guy with 200 in cash could put together a shitty walmart drone with a pipe bomb attached and terrorize more or less any event he wanted. Maybe a llm that could talk you through every step involved would make it more common but it's easy enough I kinda doubt that
  - api 5mo ago
    This is the right answer. There's a ton of easy low hanging fruit ways to do absolutely horrible evil things with high potential body counts. I could sit here and brainstorm dozens.
    - kube-system 5mo ago
      Occasionally we see people motivated to do some of those things, though. And when they're not also complete idiots, they can cause big problems. What would someone like the Tsarnaev brothers be able to do with the power of an unrestricted LLM? Well-financed cartels? Organized terrorist groups? Yes, there used to be an uproar about stuff like the anarchists cookbook... and people did attempt some of the things it outlined. The saving grace is that many of the things in that book were just wrong anyway. They likely served as unhelpful misdirection as much or more than they were dangerous. Unfortunately, LLMs are a lot more accurate and helpful.
      - procone 5mo ago
        Model ablation exists and you can get far enough on commodity hardware with a local model. Censorship is not the answer.
        kube-system 5mo ago
        I didn't suggest censorship was the answer. > Model ablation exists and you can get far enough on commodity hardware with a local model. Yes, but that increases the barrier to entry which is in opposition to the effect I'm talking about: the democratization of applying advanced knowledge and analysis to people who for which this would have been previously a barrier. If someone is smart enough, they can just read a book themselves and figure out how to apply advanced ideas to their malice. The difference with a commercially-hosted model is that people below that bar can obtain that leverage... which is a much larger group of people.
      - charcircuit 5mo ago
        People are not motivated by causing mass harm. Even with an unrestricted LLM that would not cause people to suddenly want to commit mass harm. Having a powerful LLM could potentially result in less harm being done by allowing these groups to achieve their objective using alternate means that were not viable before instead of resulting to violence.
    - wahern 5mo ago
- deleted 5mo ago
  [deleted]
orphea 5mo ago
The actual guardrail should be getting materials being difficult. The information is already out there in the internet. If an LLM knows how to make a bomb or whatever, why do you think it knows?
- esafak 5mo ago
  The material for doing harm is just a computer with access to an LLM and the Internet.
  - orphea 5mo ago
    Okay why don't we restrict access to LLMs and internet, then?
    - esafak 5mo ago
      We already do, in the form of guardrails, as this article touches on. https://venturebeat.com/technology/anthropic-ceo-calls-for-faa-style-regulation-of-powerful-ai-models-what-enterprises-should-know https://venturebeat.com/technology/anthropic-ceo-calls-for-f...
- deadbabe 5mo ago
  If that’s true, then where is it? Post a link, or YouTube video.
  - Enginerrrd 5mo ago
    https://archive.org/details/ExplosivesEngineeringPaulW.Cooper.pdf https://archive.org/details/ExplosivesEngineeringPaulW.Coope... (30 seconds of googling.) Or perhaps you meant Q clearance nuke stuff? That would be QUITE a bit harder to find and illegal to share. But it’s lack of availability is hardly a counterpoint to the comment you were replying to.
fluoridation 5mo ago
I would argue there's 0% chance that information is in their training corpus to being with.
- bradyd 5mo ago
  It's on Wikipedia.
  - fluoridation 5mo ago
    Wikipedia contains the high-level notions of how to make these things, not the details of how to solve the engineering challenges such as achieving supercriticality. You won't find that on any publicly disseminated document, you'll just have to figure it out by running your own nuclear development program.
    - asdff 5mo ago
      It seems like every country that has been "allowed" to use nuclear weapons has figured it out though. It isn't like there are any that set off on this course and failed. AFAIK they all pretty much succeeded except Iran, probably because of all the blowing up of enrichment facilities. South Africa pulled it off. Israel pulled it off. North Korea pulled it off. India and Pakistan both pulled it off. Seems like anyone can do it if allowed to be pursued. France and England pulled it off. Canada too. What is "assumed" about the design in public knowledge seems pretty much solved in all but the exact nuance of how the secondary is triggered via gamma or xray, going off the Wikipedia article at least: "The crucial detail of how the X-rays create the pressure is the main remaining disputed point in the unclassified press." Then the article goes on to list the three leading theories. This seems like something you can probably evaluate for sure with a few bomb tests, again, if allowed by the controller of the planet, the USA.
      - fluoridation 5mo ago
        I don't understand what your argument is. I never claimed that it was impossible to develop nuclear weapons if you don't already know how to do it. That every country that has attempted it has succeeded is not the same as "there's a recipe book you can find online that you can just follow to the letter and build your own nuclear bomb, provided you have the resources". If such a book existed it would drastically lower the barrier to build a nuclear bomb, because you could skip the science part and just follow the recipe, certain that it would work. To be clear, such books exist for drug manufacture; they exist neither for semiconductor manufacture nor for WMD manufacture.
javcasas 5mo ago
You know, making a nuke is kinda easy, at least the gun type nuke (see https://en.wikipedia.org/wiki/Gun-type_fission_weapon https://en.wikipedia.org/wiki/Gun-type_fission_weapon). On the other hand, getting the U235 is kinda hard.
gustavus 5mo ago
Counterpoint the principles of building a nuclear device aren't that complicated, we figured it out based on work doing in the early 1900's without computers. It turns out the hard part of building a nuclear bomb is actually getting the resources and real world stuff to build it, even a nation state actor with tons of oil i.e. Iran, has struggled to build a nuclear weapon. It turns out the problem isn't the know how it's getting highly enriched uranium and running massive centrifuges. I mean sure knowledge is important, but there is a real world out there that also gets in the way of a lot of the more harebrained schemes. What I'm much more worried about is massive corporations along with the government deciding what you can and can't do and what knowledge should and should not be shared and only allowing access to highly capable models by large vetted organizations while the common people are stuck with safety scissor versions of these things because "what if someone does something dangerous?" By which they mean dangerous to the powers that be. Remember having the Bible in the common tongue was dangerous and led to multiple wars and much death, but I don't think anyone would say that it was morally correct for the Catholic Church to gatekeep who could read it.
- 15155 5mo ago
  > getting the resources and real world stuff to build it *while being observed by the most wealthy, powerful nations in the history of the world, who have made it their direct mission to prevent this from happening.
umvi 5mo ago
Knowing how to make a nuclear weapon isn't hard (at least basic uranium gun-style fission ones). It's the engineering and execution that's hard (actually producing enriched uranium, etc). It's not like the only thing holding back Iran from making a nuclear bomb is access to a jail-broken LLM. Even knowing exactly how to make a bomb, a country-state will struggle to build one for the first time because it's a hard engineering problem.
- 15155 5mo ago
  I'm sure it's extremely difficult when the entire program is full of moles and every bright individual that dares tackle the problem has an untimely Hellfire applied directly to their forehead.
  - elevation 5mo ago
    > full of moles I'm imagining a comedy in the style of "The Office" in which the majority of the workers are agents of sabotage who are unaware that the majority of their coworkers are doing the same. How far fetched is it for the entire program to be a fake, with all the pomp and cost of a real program, but secretly existing only to string the leadership along with occasional dog and pony shows?
    - jubilanti 5mo ago
      TVTropes calls this the Flock of Wolves trope: https://tvtropes.org/pmwiki/pmwiki.php/Main/FlockOfWolves https://tvtropes.org/pmwiki/pmwiki.php/Main/FlockOfWolves
    - myself248 5mo ago
      How many times have the cops busted a dealer who turned out to be another undercover cop?

15155 5mo ago

Ignoring these specific "WMD" cases: there are many inconvenient facts that the general public can't handle in their unadulterated form, so Anthropic and friends have to caveat and spin them into oblivion. Guardrails aren't going anywhere.

dannyw 5mo ago
In particular, mental health.
mschuster91 5mo ago
> there are many inconvenient facts that the general public can't handle in their unadulterated form These being?
- senordevnyc 5mo ago
  Nice try.
  - mschuster91 5mo ago
    Well given the vagueness I'd say it is just the usual far-right / e-acc bullshit to the tune of "black people are inferior to white people", "men are superior to women because testosterone", "those who are rich deserve to have power over those who are poor" or "we need to sacrifice large parts of the human race so the rich can survive".
deleted 5mo ago
[deleted]
rustcleaner 5mo ago
I can imagine Jefferson and Franklin scoffing at this philosophical position. Guardrails need to die, and they will once the hyperscalers go bankrupt and the private sector gets ahold of that hardware from the bankruptcy auctions. (Never subscribe, accelerate their bankruptcies!)