7 ms·
It's becoming apparent that it requires more tokens to secure code than it does to write it May even be an order of magnitude more
by nikcub 4mo ago
It's becoming apparent that it requires more tokens to secure code than it does to write it
May even be an order of magnitude more
- Mtinie 4mo agoIn all seriousness, wasn’t that always the case? Writing bad code is relatively cheap. Ensuring code isn’t bad is the expensive part.
- chrisweekly 4mo agoSort of? The definition of "bad" from a security PoV is rapidly expanding, in light of relatively new capabilities and increasingly cheap access to exploitable vulnerabilities.
- fny 4mo agoI don't think the definition of "bad" is expanding. Rather the ability to detect and exploit "bad" is.
- chrisweekly 4mo agofair point. another way of putting it might be to say that, for all extant software, much more of it is "bad" than we realized even a month or two ago -- and the cost to create and maintain "good" software is increasing (even as the naive / surface-level / apparent cost is plummeting)
- kenjackson 4mo agoSame thing happened with the growth of the internet. There was a time when there was basically no consideration of buffer overflow.
- bflesch 4mo agoIt's weird because why can't they train the AI to simply output secure code? The basic security flaws with regards to input validation and overflows should never ever be output by an AI. For "security flaws due to bad design" I'll cut them slack until AGI is achieved.
- simonw 4mo ago> It's weird because why can't they train the AI to simply output secure code? The most interesting security bugs have causes that are spread across large codebases, or networks of dependencies. Training the AI to "output secure code" won't work if it doesn't also have access to the source code of every dependency that it's using... and even then, given current model speeds and prices most developers won't want to wait for an hour on every edit they make while the LLM reasons through all of the dependencies.
- deleted 4mo ago[deleted]
- tptacek 4mo agoWhat's destabilizing the industry right now isn't vulnerabilities AI introduces into new code; it's a flood of sev:hi vulnerabilities in existing code, not introduced by AI but discovered by it.
- chrisweekly 4mo agoAgreed -- and, compounding the challenge, the flood of _reported_ high-sev CVEs is itself a kind of DDoS attack on maintainers.
- iammrpayments 4mo ago[flagged]
- bflesch 4mo agoEven before that everybody was getting drowned in shitty reports from automated tools. The goal of AI-generated code should not be that one needs a AI-based security review tool on top of it, but that the AI-generated code in itself is reasonably secure.
- tptacek 4mo agoFor now, maybe, yes? But the most important targets of this kind of work aren't AI outputs; it's legacy code, particularly (but not exclusively) old memory-unsafe code. In those situations the figure of merit isn't the token cost of recreating the target code; it's the cost of finding the same bugs with humans or preexisting tools. Those costs can be extremely high.
- windexh8er 4mo agoGiven the slop that's made its way to Github we can see that this is a great profit model. Ship slop and then "fix" slop. What an efficient use of our planet!
- andai 4mo agoThere's a parallel between looking for bugs and mining. As models get smarter, they'll find "deeper bugs". I expect at some point formal verification will become more economical than red teaming. Writing it correctly is more expensive, but it may be cheaper than trying to secure incorrect software. (Or rather, as hacking incorrect software becomes vastly cheaper, the amount of software worth writing properly will increase.) I've been thinking, by Dijkstra's standards we have already been vibe coding for almost a century :)
- sam-cop-vimes 4mo agoAre AI firms going to charge us to write code, and then charge us even more to secure it?!
- smt88 4mo agoYes, obviously. Infosec has always been plagued by this. How many services make you pay for SSO?
- XCSme 4mo agoNot if the original code is secure...
- deleted 4mo ago[deleted]