4 ms·

I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did h

I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.

ares623 8mo ago
No mention of the quality of the engineers reviewing the result?
lmeyerov 8mo ago
We split our work: * Specification extraction. We have security.md and policy.md, often per module. Threat model, mechanisms, etc. This is collaborative and gets checked in for ourselves and the AI. Policy is often tricky & malleable product/business/ux decision stuff, while security is technical layers more independent of that or broader threat model. * Bug mining. It is driven by the above. It is iterative, where we keep running it to surface findings, adverserially analyze them, and prioritize them. We keep repeating until diminishing returns wrt priority levels. Likely leads to policy & security spec refinements. We use this pattern not just for security , but general bugs and other iterative quality & performance improvement flows - it's just a simple skill file with tweaks like parallel subagents to make it fast and reliable. This lets the AI drive itself more easily and in ways you explicitly care about vs noise
simonw 8mo ago
The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it is indeed a vulnerability. Coding agents can help you put that together but shouldn't be treated as infallible oracles.
- johannes1234321 8mo ago
  The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.
  - simonw 8mo ago
    Best way to figure that out is to try it and see what happens.
    - Groxx 8mo ago
      [claimed common problem exists, try X to find it] -> [Q about how to best do that] -> "the best way to do it is to do it yourself" Surely people have found patterns that work reasonably well, and it's not "everyone is completely on their own"? I get that the scene is changing fast, but that's ridiculous.
      - nl 8mo ago
        /security-review really is pretty good. But your codebase is unique. Slop in one codebase is very dangerous in another.
        LamaOfRuin 8mo ago
        For those not aware, this is a specific feature available in Claude Code. https://support.claude.com/en/articles/11932705-automated-security-reviews-in-claude-code https://support.claude.com/en/articles/11932705-automated-se...
        Groxx 8mo ago
        that's kinda what I was looking for tbh. I didn't know that was an option, and nothing in the thread (or article) seemed to imply it was. I was mostly working off "well I could ask claude to look at my code for security problems, i.e. 'plz check for security holes kthx', but is that really going to be the best option?". if "yes", then it would kinda imply that all the customization and prompt-fiddling people do is useless, which seems rather unlikely. a premade tool is a reasonable starting point.
j-conn 8mo ago
OpenAI just released “codex security”, worth trying (along with other suggestions) if your org has access https://openai.com/index/codex-security-now-in-research-preview/ https://openai.com/index/codex-security-now-in-research-prev...

ares623 8mo ago

No mention of the quality of the engineers reviewing the result?

lmeyerov 8mo ago

We split our work: * Specification extraction. We have security.md and policy.md, often per module. Threat model, mechanisms, etc. This is collaborative and gets checked in for ourselves and the AI. Policy is often tricky & malleable product/business/ux decision stuff, while security is technical layers more independent of that or broader threat model. * Bug mining. It is driven by the above. It is iterative, where we keep running it to surface findings, adverserially analyze them, and prioritize them. We keep repeating until diminishing returns wrt priority levels. Likely leads to policy & security spec refinements. We use this pattern not just for security , but general bugs and other iterative quality & performance improvement flows - it's just a simple skill file with tweaks like parallel subagents to make it fast and reliable. This lets the AI drive itself more easily and in ways you explicitly care about vs noise

simonw 8mo ago

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it is indeed a vulnerability. Coding agents can help you put that together but shouldn't be treated as infallible oracles.

johannes1234321 8mo ago
The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.
- simonw 8mo ago
  Best way to figure that out is to try it and see what happens.
  - Groxx 8mo ago
    [claimed common problem exists, try X to find it] -> [Q about how to best do that] -> "the best way to do it is to do it yourself" Surely people have found patterns that work reasonably well, and it's not "everyone is completely on their own"? I get that the scene is changing fast, but that's ridiculous.
    - nl 8mo ago
      /security-review really is pretty good. But your codebase is unique. Slop in one codebase is very dangerous in another.
      - LamaOfRuin 8mo ago
        For those not aware, this is a specific feature available in Claude Code. https://support.claude.com/en/articles/11932705-automated-security-reviews-in-claude-code https://support.claude.com/en/articles/11932705-automated-se...
      - Groxx 8mo ago
        that's kinda what I was looking for tbh. I didn't know that was an option, and nothing in the thread (or article) seemed to imply it was. I was mostly working off "well I could ask claude to look at my code for security problems, i.e. 'plz check for security holes kthx', but is that really going to be the best option?". if "yes", then it would kinda imply that all the customization and prompt-fiddling people do is useless, which seems rather unlikely. a premade tool is a reasonable starting point.

j-conn 8mo ago

OpenAI just released “codex security”, worth trying (along with other suggestions) if your org has access https://openai.com/index/codex-security-now-in-research-preview/ https://openai.com/index/codex-security-now-in-research-prev...