4 ms·

The Malicious Use of Artificial Intelligence

87 points by rasengan0 15d ago

voidhorse 15d ago
Should probably have a (2018) or (2024) (latest revision) on the title, especially given the current buzz surrounding AI and security/existential threats.
EGreg 15d ago
This paper diagnosed the disease in 2018. Eight years later none of the recommendations happened. Norms, collaboration, responsible disclosure - none of it materialized in any structural way. I think the reason is that the paper frames malicious AI use as a policy problem, recommending social solutions. It's actually an architecture problem. Every generation of computing has hit a version of this. Programs could write anywhere in memory - we added protected memory. Programs could hog the CPU - we added preemptive multitasking. Desktop apps could call any OS function - the iPhone sandboxed them. Nobody asked programs to please behave, the containment actually went into the infrastructure. AI skipped that step entirely. We went straight to open-ended agents with broad permissions and tried to make them safe through alignment and prompting. I've been researching this for the past year and I think alignment is necessary but not sufficient, because the intelligence increasingly isn't in the model. It's in the substrate - the harness, the domain knowledge, the tooling around the model. I actually measured this on real coding tasks: Sonnet with a code-derived index outperformed the frontier model (Opus 5.8) exploring on its own, and the top-tier model (Fable) refused the real work entirely! The cheap model with the right rig beat the expensive model without one. https://safebots.ai/matchup.html https://safebots.ai/matchup.html If that's true then aligning the model doesn't solve the problem. A bad actor who can't get the best model just uses Sonnet. Or Llama. Or Kimi. The weights have already leaked and bits don't degrade - you can't recall them the way you can stop manufacturing CFCs. So what do you actually do? Same thing that worked for CFCs. You gotta first build the safe version — in this case, declarative workflows running in sealed compute environments — and prove it handles 99% of actual use cases at lower cost. Let it win commercially. Then regulate the dangerous version. DuPont developed HFC refrigerants first. The Montreal Protocol became possible BECAUSE of that. The ban became politically viable because the alternative already existed. I've been building this alternative for the past 8 months: https://safebots.ai/about https://safebots.ai/about
- esafak 15d ago
  Alignment is the architectural solution. Make it so the model can't misbehave. Yet many people here deride it as tainting the model, claiming "Whose values is it aligned to?" Sandboxes are a last ditch layer. They fail, as we see.
  - _vertigo 15d ago
    > Make it so the model can't misbehave. How do you figure? I haven't met anyone who thinks that's possible. It seems clear to me that it is not possible.
    - esafak 15d ago
      Embed a constitution they can't override. Project bad outputs to their nearest acceptable one. If we have to stop model development to ensure we can do it, so be it.
      - dgellow 14d ago
        I agree, but because I want to see the development stopped forever, which is what the result of this would be. You will never have alignment that cannot be overridden in some ways. You won’t have a silver bullet here, you need safety at every layer
  - startup_zombie_ 15d ago
    The “make it so the model can't misbehave” part is interesting. Maybe the goal isn't to make the model perfectly aligned, but to make misalignment have a very small blast radius. That feels like a more achievable engineering problem.
  - root_axis 15d ago
    > Make it so the model can't misbehave Not possible. They can chase the models with whack a mole tuning for obvious stuff, but there's always a way to extract what you want from the model.
  - 15d ago
deleted 15d ago
[deleted]
wartywhoa23 14d ago
The most malicious use of AI would be using it as a cover for the elites to perpetrate stuff that is way out of the current Overton window, and blame AI for that.
- Cthulhu_ 14d ago
  The solution is simple, hold them responsible. It's software, and people running and developing software are responsible for what it does. If I write a virus I'll get the various cybercrime units on me. The LLM companies don't get a pass if their software does something malicious. They shouldn't get to hide behind "we don't know what it's doing!" (have you tried looking and monitoring?) or "it's too powerful!" (have you tried turning it off when you realised it was doing something bad?)
  - grafmax 14d ago
    I wish. The government, regardless of party, wants AI for mass surveillance and perpetual war. It is paid off and lobbied and effectively another arm of the billionaires. Holding big companies responsible for their actions won't happen as long as big companies control the political apparatus. Like, when have we ever seen big companies receive more than a slap on the wrist? They are basically free to do anything, hack, murder, crash the economy, perpetuate genocide. That's why something is a crime if you or I were to do it but not a crime when these companies do so. But never fear, they will constantly tell us how what they're doing is for the sake of democracy or safety or ethics or whatever the doublethink of the day is.
- m4xp 14d ago
  Ai will 100% be use maliciously, people will already think It's all knowing, meanwhile any expert in any field cringes at this notion. Ai will be used as justification for something, oh ai said this? It must be true, lets ignore how tge model was trained for this outcome and the secret promp that is not disclosed.
ramon156 14d ago
If anyone accidentally DDoS-es their uni, they can just say claude did it and get away with it, right? right?
titaniumrain 14d ago
too many ppl whining about AI

87 points by rasengan0 15d ago

Should probably have a (2018) or (2024) (latest revision) on the title, especially given the current buzz surrounding AI and security/existential threats.

EGreg 15d ago

This paper diagnosed the disease in 2018. Eight years later none of the recommendations happened. Norms, collaboration, responsible disclosure - none of it materialized in any structural way. I think the reason is that the paper frames malicious AI use as a policy problem, recommending social solutions. It's actually an architecture problem. Every generation of computing has hit a version of this. Programs could write anywhere in memory - we added protected memory. Programs could hog the CPU - we added preemptive multitasking. Desktop apps could call any OS function - the iPhone sandboxed them. Nobody asked programs to please behave, the containment actually went into the infrastructure. AI skipped that step entirely. We went straight to open-ended agents with broad permissions and tried to make them safe through alignment and prompting. I've been researching this for the past year and I think alignment is necessary but not sufficient, because the intelligence increasingly isn't in the model. It's in the substrate - the harness, the domain knowledge, the tooling around the model. I actually measured this on real coding tasks: Sonnet with a code-derived index outperformed the frontier model (Opus 5.8) exploring on its own, and the top-tier model (Fable) refused the real work entirely! The cheap model with the right rig beat the expensive model without one. https://safebots.ai/matchup.html https://safebots.ai/matchup.html If that's true then aligning the model doesn't solve the problem. A bad actor who can't get the best model just uses Sonnet. Or Llama. Or Kimi. The weights have already leaked and bits don't degrade - you can't recall them the way you can stop manufacturing CFCs. So what do you actually do? Same thing that worked for CFCs. You gotta first build the safe version — in this case, declarative workflows running in sealed compute environments — and prove it handles 99% of actual use cases at lower cost. Let it win commercially. Then regulate the dangerous version. DuPont developed HFC refrigerants first. The Montreal Protocol became possible BECAUSE of that. The ban became politically viable because the alternative already existed. I've been building this alternative for the past 8 months: https://safebots.ai/about https://safebots.ai/about

esafak 15d ago
Alignment is the architectural solution. Make it so the model can't misbehave. Yet many people here deride it as tainting the model, claiming "Whose values is it aligned to?" Sandboxes are a last ditch layer. They fail, as we see.
- _vertigo 15d ago
  > Make it so the model can't misbehave. How do you figure? I haven't met anyone who thinks that's possible. It seems clear to me that it is not possible.
  - esafak 15d ago
    Embed a constitution they can't override. Project bad outputs to their nearest acceptable one. If we have to stop model development to ensure we can do it, so be it.
    - dgellow 14d ago
      I agree, but because I want to see the development stopped forever, which is what the result of this would be. You will never have alignment that cannot be overridden in some ways. You won’t have a silver bullet here, you need safety at every layer
- startup_zombie_ 15d ago
  The “make it so the model can't misbehave” part is interesting. Maybe the goal isn't to make the model perfectly aligned, but to make misalignment have a very small blast radius. That feels like a more achievable engineering problem.
- root_axis 15d ago
  > Make it so the model can't misbehave Not possible. They can chase the models with whack a mole tuning for obvious stuff, but there's always a way to extract what you want from the model.
- 15d ago

deleted 15d ago

[deleted]

wartywhoa23 14d ago

The most malicious use of AI would be using it as a cover for the elites to perpetrate stuff that is way out of the current Overton window, and blame AI for that.

Cthulhu_ 14d ago
The solution is simple, hold them responsible. It's software, and people running and developing software are responsible for what it does. If I write a virus I'll get the various cybercrime units on me. The LLM companies don't get a pass if their software does something malicious. They shouldn't get to hide behind "we don't know what it's doing!" (have you tried looking and monitoring?) or "it's too powerful!" (have you tried turning it off when you realised it was doing something bad?)
- grafmax 14d ago
  I wish. The government, regardless of party, wants AI for mass surveillance and perpetual war. It is paid off and lobbied and effectively another arm of the billionaires. Holding big companies responsible for their actions won't happen as long as big companies control the political apparatus. Like, when have we ever seen big companies receive more than a slap on the wrist? They are basically free to do anything, hack, murder, crash the economy, perpetuate genocide. That's why something is a crime if you or I were to do it but not a crime when these companies do so. But never fear, they will constantly tell us how what they're doing is for the sake of democracy or safety or ethics or whatever the doublethink of the day is.
m4xp 14d ago
Ai will 100% be use maliciously, people will already think It's all knowing, meanwhile any expert in any field cringes at this notion. Ai will be used as justification for something, oh ai said this? It must be true, lets ignore how tge model was trained for this outcome and the secret promp that is not disclosed.

ramon156 14d ago

If anyone accidentally DDoS-es their uni, they can just say claude did it and get away with it, right? right?

titaniumrain 14d ago

too many ppl whining about AI