4 ms·

> Closed source software won't receive any reports, but it will be exploited with AI. What makes you so sure that closed-source companies won't run those same

> Closed source software won't receive any reports, but it will be exploited with AI. What makes you so sure that closed-source companies won't run those same AI scanners on their own code? It's closed to the public, it's not closed to them!

LunicLynx 7mo ago
Came here to say the same. Same tools + private. In security two different defense-mechanisms are always better than one.
- bluebarbet 7mo ago
  Same tools A, B and C, but minus tools D, E and F, and with a smaller chance that any tools at all will even be used. Not claiming that it's a slam dunk for open source, but the inverse does not seem correct either.
  - LunicLynx 7mo ago
    Fair enough
  - lelanthran 7mo ago
    > Same tools A, B and C, but minus tools D, E and F, Why "minus D, E and F"? After all, once you have the harness set up, there's no additional work to add in new models, right?
    - bluebarbet 7mo ago
      The point being that there are always going to be more eyes, and more knowledge of available tools (i.e. including "D, E and F"), and more experience using them, with open source than with a single in-house dev team.
      - lelanthran 7mo ago
        There's no more "eyes" though, it's all models, and they are all converging pretty damn fast.
        bluebarbet 7mo ago
        If true then logically it will be sufficient to run this "master model" once before any code release for the level playing field to be restored. After all, even open-source software is private until it is released.
        lelanthran 7mo ago
        > If true then logically it will be sufficient to run this "master model" once before any code release for the level playing field to be restored. I'm struggling to see how it is a level playing field: 1. Closed-source: defender runs llms to check the sources for vulns, runs llms on each PR, runs llm on deployment of the compiled output. Attacker runs llm only on compiled output. 2. Open-source: both attacker and defender runs llms on source, on PRs and on compiled output.
ihaveajob 7mo ago
More eyes, more chances that someone will actually use the tools. Also, the tools and how you use them are not all the same.
- phendrenad2 7mo ago
  With enough copies of GPT printing out the same bulleted list, all bugs are 1. shallow 2. hollow 3. flat ...
440bx 7mo ago
As someone who works on closed source software and has done for a couple of decades, most companies won't even know about that and of those who do only a fraction give enough of a shit about it to do anything until they are caught with their pants down.
- sdoering 7mo ago
  Seconded. Having worked in quite a few agency/consultancy situations, it is far more productive to smash your head against a wall till bleeding, than to get a client to pay for security. The regular answer: "This is table stakes, we pay you for this." Combined with: "Why has velocity gone down, we don't pay you for that security or documentation crap." There are unexploited security holes in enterprise software you can drive a boring machine through. There is a well paid "security" (aka employee surveillance) company using python2.7 (no, not patched) on each and every machine their software runs on. At some of the biggest companies in this world. They just don't care for updating this, because, why should they. There is no incentive. None.
  - valeriozen 7mo ago
    Yea, its fundamentally an issue of asymmetric economics. Running AI scanners internally costs money, dev time, and management buy in to actually fix the mountain of tech debt the scanners uncover. As you said there is no incentive for that But for bad actors the cost of pointing an LLM at an exposed endpoint or reverse engineered binary has dropped to near zero. The attackers tooling just got exponentially cheaper and faster, while the enterprise defenders budget remained at zero.
    - njyx 7mo ago
      In theory though, there is now a new way for community to support open source, but running vulnerability scans in white-hat mode, reporting and patching. That way they burn tokens for a project they love. Even if they couldn't actually contribute code before. There should be a way to donate your unused tokens on every cycle to open source like rounding up at the chekout!
baileypumfleet 7mo ago
As I mentioned above, we actually do run these AI scanners on our code, but the problem is it's simply not enough. These AI scanners, including STRIX, don't find everything. Each scanning tool actually finds different results from the other, and so it's impossible to determine a benchmark of what's secure and what's not.
- topopopo 7mo ago
  I think it makes it all the more apparent that writing EAL4 code with as little design competence as possible was taking advantage of some strange scarcity economics.. It's now even easier to make something with endless technical debt and security vs backwards compatibility liability but is anyone going to keep paying for things that aren't correct and to the point if some market participants structure their agent usage toward verifiable quality and don't actually have more cost any more?
- lelanthran 7mo ago
  > As I mentioned above, we actually do run these AI scanners on our code, but the problem is it's simply not enough. These AI scanners, including STRIX, don't find everything. Each scanning tool actually finds different results from the other, and so it's impossible to determine a benchmark of what's secure and what's not. Yeah, but with closed source it's cheaper for the defender than for the attacker - the defender can scan their sources and their PRs as well as the compiled output. The attacker can only scan the compiled output, and they have to perform repeated scans.
suhputt 7mo ago
[dead]
cyanydeez 7mo ago
Because they're a company. Even if the bar to entry can fit a normal sized american, doesn't mean they will do it, or do it in a systematic way; We know very well that nothing about AI is naturally systematic, so why would you assume it'll happen in a systematic way.
dspillett 7mo ago
> What makes you so sure that closed-source companies won't run those same AI scanners on their own code? How many companies take the time to use penetration testing tools, that have been available for many years, to verify their software (or pay a penetration testing company to do a more thorough job than they have the experience to do internally)? Some, certainly. Many, possibly. Most, I would wager not.
necheffa 7mo ago
The economic motivation just simply isn't there. I'm sure we could cherry pick a few examples of companies where things like quality and security really are part of the culture, not just feel-good lip services. The reality is that companies are in business to make money and corner cutting is the easiest way to pad the margins.

LunicLynx 7mo ago

Came here to say the same. Same tools + private. In security two different defense-mechanisms are always better than one.

bluebarbet 7mo ago
Same tools A, B and C, but minus tools D, E and F, and with a smaller chance that any tools at all will even be used. Not claiming that it's a slam dunk for open source, but the inverse does not seem correct either.
- LunicLynx 7mo ago
  Fair enough
- lelanthran 7mo ago
  > Same tools A, B and C, but minus tools D, E and F, Why "minus D, E and F"? After all, once you have the harness set up, there's no additional work to add in new models, right?
  - bluebarbet 7mo ago
    The point being that there are always going to be more eyes, and more knowledge of available tools (i.e. including "D, E and F"), and more experience using them, with open source than with a single in-house dev team.
    - lelanthran 7mo ago
      There's no more "eyes" though, it's all models, and they are all converging pretty damn fast.
      - bluebarbet 7mo ago
        If true then logically it will be sufficient to run this "master model" once before any code release for the level playing field to be restored. After all, even open-source software is private until it is released.
        lelanthran 7mo ago
        > If true then logically it will be sufficient to run this "master model" once before any code release for the level playing field to be restored. I'm struggling to see how it is a level playing field: 1. Closed-source: defender runs llms to check the sources for vulns, runs llms on each PR, runs llm on deployment of the compiled output. Attacker runs llm only on compiled output. 2. Open-source: both attacker and defender runs llms on source, on PRs and on compiled output.

ihaveajob 7mo ago

More eyes, more chances that someone will actually use the tools. Also, the tools and how you use them are not all the same.

phendrenad2 7mo ago
With enough copies of GPT printing out the same bulleted list, all bugs are 1. shallow 2. hollow 3. flat ...

440bx 7mo ago

As someone who works on closed source software and has done for a couple of decades, most companies won't even know about that and of those who do only a fraction give enough of a shit about it to do anything until they are caught with their pants down.

sdoering 7mo ago
Seconded. Having worked in quite a few agency/consultancy situations, it is far more productive to smash your head against a wall till bleeding, than to get a client to pay for security. The regular answer: "This is table stakes, we pay you for this." Combined with: "Why has velocity gone down, we don't pay you for that security or documentation crap." There are unexploited security holes in enterprise software you can drive a boring machine through. There is a well paid "security" (aka employee surveillance) company using python2.7 (no, not patched) on each and every machine their software runs on. At some of the biggest companies in this world. They just don't care for updating this, because, why should they. There is no incentive. None.
- valeriozen 7mo ago
  Yea, its fundamentally an issue of asymmetric economics. Running AI scanners internally costs money, dev time, and management buy in to actually fix the mountain of tech debt the scanners uncover. As you said there is no incentive for that But for bad actors the cost of pointing an LLM at an exposed endpoint or reverse engineered binary has dropped to near zero. The attackers tooling just got exponentially cheaper and faster, while the enterprise defenders budget remained at zero.
  - njyx 7mo ago
    In theory though, there is now a new way for community to support open source, but running vulnerability scans in white-hat mode, reporting and patching. That way they burn tokens for a project they love. Even if they couldn't actually contribute code before. There should be a way to donate your unused tokens on every cycle to open source like rounding up at the chekout!

baileypumfleet 7mo ago

As I mentioned above, we actually do run these AI scanners on our code, but the problem is it's simply not enough. These AI scanners, including STRIX, don't find everything. Each scanning tool actually finds different results from the other, and so it's impossible to determine a benchmark of what's secure and what's not.

topopopo 7mo ago
I think it makes it all the more apparent that writing EAL4 code with as little design competence as possible was taking advantage of some strange scarcity economics.. It's now even easier to make something with endless technical debt and security vs backwards compatibility liability but is anyone going to keep paying for things that aren't correct and to the point if some market participants structure their agent usage toward verifiable quality and don't actually have more cost any more?
lelanthran 7mo ago
> As I mentioned above, we actually do run these AI scanners on our code, but the problem is it's simply not enough. These AI scanners, including STRIX, don't find everything. Each scanning tool actually finds different results from the other, and so it's impossible to determine a benchmark of what's secure and what's not. Yeah, but with closed source it's cheaper for the defender than for the attacker - the defender can scan their sources and their PRs as well as the compiled output. The attacker can only scan the compiled output, and they have to perform repeated scans.

suhputt 7mo ago

[dead]

cyanydeez 7mo ago

Because they're a company. Even if the bar to entry can fit a normal sized american, doesn't mean they will do it, or do it in a systematic way; We know very well that nothing about AI is naturally systematic, so why would you assume it'll happen in a systematic way.

dspillett 7mo ago

> What makes you so sure that closed-source companies won't run those same AI scanners on their own code? How many companies take the time to use penetration testing tools, that have been available for many years, to verify their software (or pay a penetration testing company to do a more thorough job than they have the experience to do internally)? Some, certainly. Many, possibly. Most, I would wager not.

necheffa 7mo ago

The economic motivation just simply isn't there. I'm sure we could cherry pick a few examples of companies where things like quality and security really are part of the culture, not just feel-good lip services. The reality is that companies are in business to make money and corner cutting is the easiest way to pad the margins.