4 ms·
After a release, attackers have effectively infinite time to throw an LLM against every line of your code - an LLM that only gets smarter and cheaper to run as
by johnfn 6mo ago
After a release, attackers have effectively infinite time to throw an LLM against every line of your code - an LLM that only gets smarter and cheaper to run as time passes. In order to feel secure you’d need to do all the work you’d imagine an attacker would ever do, for every single release you ship.
- stavros 6mo agoThis assumes that the relationship between "LLM tokens spent" and "vulnerabilities found" doesn't plateau, though.
- mixdup 6mo agoThe first few times it's going to be expensive, but once everyone level sets with intense scans of their codebases, "every single release" is actually not that big a deal, since you are not likely to be completely rebuilding your codebase every release
- techpression 6mo agoYou still have to account for the non-deterministic behavior of an LLM, when do you know you have exhausted its possible outcomes for any given piece of code?
- johnfn 6mo agoI'm not sure. An innocuous one line change like "bump version" possibly adds a million new lines of code.
- utopiah 6mo ago> attackers have effectively infinite time No, attackers are also rational economical actors. They don't randomly attack any software just for the aesthetics beauty of the process. They attack for bounty, for fame, for national interest, etc. No matter the reason it's not random and thus they DO have a budget, both in time and money. They attack THIS project versus another project because it's interesting to them. If it's not, they might move to another project but they certainly won't spend infinite time precisely because they don't have infinite resources. IMHO it's much more interesting to consider the realistic arm race then theoretical scenarii that never take place.
- johnfn 6mo agoThe amount of time they will invest is proportional to how much usage / how high value the target is. If your release is used by no one then no one is going to attack it, but it didn't matter anyways.
- rhubarbtree 6mo agoBut so do you and all your users what’s your point?
- johnfn 6mo agoNo? I can't go out and retroactively fix a bug in a version my users are using? I need to release a new version?