4 ms·

The golden age of net security is here... Both the defense is weaker due to LLMs and attacks become stronger and cheaper. Bad combination for the rest of us.

embedding-shape 5mo ago
> Both the defense is weaker due to LLMs and attacks become stronger Are you claiming that LLMs are better at offensive security than defensive security? Or somehow that the offensive actors have access to better LLMs than people using them to defend? Otherwise it'd seem like the playing field just went up for both sides, unless one is famously lagging behind because no like to pay for better security? But that's also nothing new.
- YZF 5mo ago
  The question is whether LLMs write more secure code than humans. If we get a lot of vibe coded software coming online by non-SWEs do we think that would be more or less secure?
- teaearlgraycold 5mo ago
  Defense is weaker because of vibe coding. Computer security is asymmetric. Attacking is easier than defending. Attackers need to find one hole in the security. Defenders need to patch every hole.
- overgard 5mo ago
  Even if defense keeps up it kind of depends on entities keeping up to date.. complex software stacks can make that hard, or falling behind a major version, etc. I think defense is harder than offense in this era
- deleted 5mo ago
  [deleted]
- jesse_ash 5mo ago
  IMO the assumption is probably that, with LLMs generally, software complexity and surface area going up faster than we're tackling it through hardening, testing, etc. - even with the help of defensive models. I would also imagine bad actors are in the majority, and so we're seeing restrictions on models like Mythos in an attempt to balance the field a bit.
- dalmo3 5mo ago
  It's order vs chaos, and LLMs are on the side of chaos.
- amarant 5mo ago
  He's just karma farming with the ever so creative "LLM=bad" hot take.. I don't know what's sadder: that people are doing that on HN, or that it's clearly working....
- hansvm 5mo ago
  Ignoring LLMs, the status quo for defense is that you're pwnable from the silliest of mistakes, and the status quo for offense is that even one lucky shot lets you in. Suppose you brought in 1000x more people to projects on both sides; you'd expect a much higher chance of at least one failure for the defenders and at least one success for the attackers. LLMs don't have the same dynamics, but the same underlying idea is worth bearing in mind. Above and beyond that, yes, defense is harder for LLMs than offense. They struggle mightily when pulling together too many threads, and some projects are just too big. On the defensive side, exploits are usually very tiny and asymmetrically acceleratable via LLMs.
  - leereeves 5mo ago
    > defense is harder for LLMs than offense. They struggle mightily when pulling together too many threads, and some projects are just too big. Defense doesn't have to be part of code gen. Any automated attacker (like LLMs) could be used for defense simply by finding exploits, then fixing them by whatever means. People choose not to invest in that.
- LPisGood 5mo ago
  I don’t think it matters so much if LOM are better at offensive security or defensive security. I think offensive security was previously an extremely niche skill set (how many people off the street would be able to solve even a few CTF problems 5 years ago?). Now anyone can point an LLM at any software they want and say go to town. Even if it doesn’t do a great job or better than a good human or anything like that it’s so much more than what they could do before, and a lot of security vulnerabilities are kind of low hanging fruit anyway.
  - embedding-shape 5mo ago
    But if that's true for attacking, I don't see how it's not also true for defence? If LLMs make it easier to attack, and this is a generally available model, then it can as well be used for defence, and easier too, since you have the source too.
Reaktornano 5mo ago
I personally do not think that the defence of any particular project is weaker, but the overall internet as a bunch of interdependencies is much weaker, as you never know which open source library in depths of code was compromized
- thierrydamiba 5mo ago
  The scary thing for me is most of the vulnerabilities have been revealed due to silly mistakes. If someone really knew what they were doing and had bad intentions, I fear we would never find out.
dyauspitr 5mo ago
Does anyone LLM bad?

embedding-shape 5mo ago

> Both the defense is weaker due to LLMs and attacks become stronger Are you claiming that LLMs are better at offensive security than defensive security? Or somehow that the offensive actors have access to better LLMs than people using them to defend? Otherwise it'd seem like the playing field just went up for both sides, unless one is famously lagging behind because no like to pay for better security? But that's also nothing new.

YZF 5mo ago
The question is whether LLMs write more secure code than humans. If we get a lot of vibe coded software coming online by non-SWEs do we think that would be more or less secure?
teaearlgraycold 5mo ago
Defense is weaker because of vibe coding. Computer security is asymmetric. Attacking is easier than defending. Attackers need to find one hole in the security. Defenders need to patch every hole.
overgard 5mo ago
Even if defense keeps up it kind of depends on entities keeping up to date.. complex software stacks can make that hard, or falling behind a major version, etc. I think defense is harder than offense in this era
deleted 5mo ago
[deleted]
jesse_ash 5mo ago
IMO the assumption is probably that, with LLMs generally, software complexity and surface area going up faster than we're tackling it through hardening, testing, etc. - even with the help of defensive models. I would also imagine bad actors are in the majority, and so we're seeing restrictions on models like Mythos in an attempt to balance the field a bit.
dalmo3 5mo ago
It's order vs chaos, and LLMs are on the side of chaos.
amarant 5mo ago
He's just karma farming with the ever so creative "LLM=bad" hot take.. I don't know what's sadder: that people are doing that on HN, or that it's clearly working....
hansvm 5mo ago
Ignoring LLMs, the status quo for defense is that you're pwnable from the silliest of mistakes, and the status quo for offense is that even one lucky shot lets you in. Suppose you brought in 1000x more people to projects on both sides; you'd expect a much higher chance of at least one failure for the defenders and at least one success for the attackers. LLMs don't have the same dynamics, but the same underlying idea is worth bearing in mind. Above and beyond that, yes, defense is harder for LLMs than offense. They struggle mightily when pulling together too many threads, and some projects are just too big. On the defensive side, exploits are usually very tiny and asymmetrically acceleratable via LLMs.
- leereeves 5mo ago
  > defense is harder for LLMs than offense. They struggle mightily when pulling together too many threads, and some projects are just too big. Defense doesn't have to be part of code gen. Any automated attacker (like LLMs) could be used for defense simply by finding exploits, then fixing them by whatever means. People choose not to invest in that.
LPisGood 5mo ago
I don’t think it matters so much if LOM are better at offensive security or defensive security. I think offensive security was previously an extremely niche skill set (how many people off the street would be able to solve even a few CTF problems 5 years ago?). Now anyone can point an LLM at any software they want and say go to town. Even if it doesn’t do a great job or better than a good human or anything like that it’s so much more than what they could do before, and a lot of security vulnerabilities are kind of low hanging fruit anyway.
- embedding-shape 5mo ago
  But if that's true for attacking, I don't see how it's not also true for defence? If LLMs make it easier to attack, and this is a generally available model, then it can as well be used for defence, and easier too, since you have the source too.

Reaktornano 5mo ago

I personally do not think that the defence of any particular project is weaker, but the overall internet as a bunch of interdependencies is much weaker, as you never know which open source library in depths of code was compromized

thierrydamiba 5mo ago
The scary thing for me is most of the vulnerabilities have been revealed due to silly mistakes. If someone really knew what they were doing and had bad intentions, I fear we would never find out.

dyauspitr 5mo ago

Does anyone LLM bad?