3 ms·
The end result of that is that your model can't fix or acknowledge security issues for fear of disclosing them. This is the beauty the above poster mentioned:
by torben-friis 4mo ago
The end result of that is that your model can't fix or acknowledge security issues for fear of disclosing them.
This is the beauty the above poster mentioned: the ability to improve code is inherently coupled with the ability to recognize its shortcomings. You can't have one without the other.
- btilly 4mo agoWhat I suggested would allow it to fix the issues. Just not write a test that was directly usable as a security exploit. This doesn't stop attackers from being able to leverage the analysis. But it does make the tool more useful for defenders than attackers. Which is the best that you can hope for from a useful tool.
- torben-friis 4mo agoIt hides the issue a bit. But if you ask for atomic security fixes and then stare at the diffs you have your vulnerability. There is just a bit more friction involved in the vulnerability => exploit path, but the root cause is unfixed. I think it even might be possible to route the isolated fix somewhere to automate that last step. Maybe invert the diff and pass it through automated code review for example, see the reasoning when the llm flags the change as dangerous.
- Marsymars 4mo ago> What I suggested would allow it to fix the issues. Just not write a test that was directly usable as a security exploit. It will be pretty obvious what are security issues in that case - i.e. all the code changes that don't have corresponding tests.