2 ms·

> to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them. That can't be right, can it? Given stable so

> to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them. That can't be right, can it? Given stable software, the relative attack surface keeps shrinking. Mythos does not produce exploits. Should be defenders advantage, token wise, no?

paisawalla 6mo ago
So long as that OSS keeps accumulating features, there isn't quite the equilibrium you're imagining. If you can pin to a stable version, which continues to audited, you're fine. But if the rest of the world moves on to newer versions of the software, you'll have to as well, unless you want to own the burden of hardening older versions.
rhplus 6mo ago
It’s the classic asymmetric warfare problem: Defenders have to find all the holes in all their systems, while attackers just need to find one hole in one system.
- lexlambda 6mo ago
  A slight factor differentiating security systems here is involved to the advantage of defenders: Attackers have to find a whole exploit chain, while defenders only need to fix one part of it.
- jstummbillig 6mo ago
  The point is that, as the defender, you only have to find each hole once, while the attacker can spend an infinite amount of tokens trying to find more holes, that are increasingly harder to find and might, eventually, not exist at all. The defender can do that too, of course, but being in the defense, there is value in not being able to uncover new holes (your system keeps working, ostensibly) while as the attacker that's simply how you fail.
JoshTriplett 6mo ago
> Mythos does not produce exploits. AI in general will, don't worry. "Move fast and break things" makes more exploits than "move steadily and fix things" does.
- Gormo 6mo ago
  But doesn't AI ultimately obviate "move fast and break things" by making it easier to move fast without breaking things?
  - JoshTriplett 6mo ago
    Not at all, no; AI makes it harder to not break things, and it takes a lot of work to not break things.
    - Gormo 6mo ago
      It does take a lot of work not to break things. That's why "move fast and break things" are traditionally closely coupled: it's hard to avoid breaking things without slowing down. But why would responsible AI users -- actual engineers using it to accelerate grunt work, not vibe coders -- not use the AI tooling to increase their capacity to do all of the work it takes to avoid breaking things while still moving fast, relatively speaking? Testing a new incremental feature against the entire extant codebase, not just the bits of it that they had the bandwidth to tackle within the deadline, seems like exactly the sort of thing well-disciplined engineering teams would use AI to do.
      - habinero 6mo ago
        There's a couple points of misunderstanding here. For one, you architect your codebase into separate layers and logical chunks that are self-contained and can be reasoned about independently. That's not always possible, but you draw as many firm boundaries as you can. You don't ever want to be in the position where you have to test an entire codebase against your new change. That's a horrible nightmare scenario. So you don't "test as much of the codebase as you have time for", you write tests for your code and the interface between it and other systems. Maybe integration or FE tests depending on what you have. So testing against a whole codebase is rarely the problem, and if it is, you have bigger issues. Also, LLMs don't make mistakes like humans do. They fuck up in weird unpredictable ways that mean you kinda have to treat them like a hostile adversary trying to sneak in subtle backdoors. It slows things down. Also, actually writing code is usually the fast and easy part. It's all the other bits -- getting the requirements, building mockups, planning, review, standing up new infra etc etc etc. LLMs can't help with most of that.

paisawalla 6mo ago

So long as that OSS keeps accumulating features, there isn't quite the equilibrium you're imagining. If you can pin to a stable version, which continues to audited, you're fine. But if the rest of the world moves on to newer versions of the software, you'll have to as well, unless you want to own the burden of hardening older versions.

rhplus 6mo ago

It’s the classic asymmetric warfare problem: Defenders have to find all the holes in all their systems, while attackers just need to find one hole in one system.

lexlambda 6mo ago
A slight factor differentiating security systems here is involved to the advantage of defenders: Attackers have to find a whole exploit chain, while defenders only need to fix one part of it.
jstummbillig 6mo ago
The point is that, as the defender, you only have to find each hole once, while the attacker can spend an infinite amount of tokens trying to find more holes, that are increasingly harder to find and might, eventually, not exist at all. The defender can do that too, of course, but being in the defense, there is value in not being able to uncover new holes (your system keeps working, ostensibly) while as the attacker that's simply how you fail.

JoshTriplett 6mo ago

> Mythos does not produce exploits. AI in general will, don't worry. "Move fast and break things" makes more exploits than "move steadily and fix things" does.

Gormo 6mo ago
But doesn't AI ultimately obviate "move fast and break things" by making it easier to move fast without breaking things?
- JoshTriplett 6mo ago
  Not at all, no; AI makes it harder to not break things, and it takes a lot of work to not break things.
  - Gormo 6mo ago
    It does take a lot of work not to break things. That's why "move fast and break things" are traditionally closely coupled: it's hard to avoid breaking things without slowing down. But why would responsible AI users -- actual engineers using it to accelerate grunt work, not vibe coders -- not use the AI tooling to increase their capacity to do all of the work it takes to avoid breaking things while still moving fast, relatively speaking? Testing a new incremental feature against the entire extant codebase, not just the bits of it that they had the bandwidth to tackle within the deadline, seems like exactly the sort of thing well-disciplined engineering teams would use AI to do.
    - habinero 6mo ago
      There's a couple points of misunderstanding here. For one, you architect your codebase into separate layers and logical chunks that are self-contained and can be reasoned about independently. That's not always possible, but you draw as many firm boundaries as you can. You don't ever want to be in the position where you have to test an entire codebase against your new change. That's a horrible nightmare scenario. So you don't "test as much of the codebase as you have time for", you write tests for your code and the interface between it and other systems. Maybe integration or FE tests depending on what you have. So testing against a whole codebase is rarely the problem, and if it is, you have bigger issues. Also, LLMs don't make mistakes like humans do. They fuck up in weird unpredictable ways that mean you kinda have to treat them like a hostile adversary trying to sneak in subtle backdoors. It slows things down. Also, actually writing code is usually the fast and easy part. It's all the other bits -- getting the requirements, building mockups, planning, review, standing up new infra etc etc etc. LLMs can't help with most of that.