4 ms·
The world is so not ready for the impact of LLMs on security issues. If true, congrats to the Calif team. It’s likely too technical for me to understand in deta
by dgellow 6mo ago
The world is so not ready for the impact of LLMs on security issues. If true, congrats to the Calif team. It’s likely too technical for me to understand in details but looking forward to reading the 55 pages report
- iqihs 6mo agoyou're assuming that blue teams and engineers are sitting around twiddling their thumbs
- dgellow 6mo agoNot at all. I’m considering that the amount of vulnerable software in the wild is very, very large, with most organizations not managing their systems properly. Imagine all the small to medium size companies that do not have budgets for a dedicated, talented security team. And all the software that will never be patched. We are at the beginning of the exponential
- bottlepalm 6mo agoIt makes you think will everything need to be rewritten from the ground up - potentially by AI itself, or AI having a very heavy hand in validating all of it.
- Gigachad 6mo agoThere's so much much lower hanging fruit. Every job I've had has had basically everything massively out of date. Just keeping packages and framework versions up to date is a full time job and none of these companies have someone assigned to doing it. So much out of date software with known exploits left running for years. The only reason there hasn't been total disaster is no one has tried to hack it yet.
- bottlepalm 6mo agoRight and with AI now we have the ability to try hacking everything all at once.
- dgellow 6mo agoYes, exactly, that’s the main change. And not just in a script kiddy way. What we see now is LLM + experts can develop extremely complex exploit chains in no time. It’s one thing to exploit a known vulnerability that you can patch by upgrading your Wordpress, it’s something else when the attacker is able to completely take over your systems in ways you didn’t even consider was possible and adapt in 1 day to your attempts at patching
- Gigachad 6mo agoFor now, after the dust settles all of the low hanging fruit will have been patched and we will have hurried up the move to safer languages. The root problem is the world runs on C code that is riddled with vulnerabilities.
- saghm 6mo ago> I’m considering that the amount of vulnerable software in the wild is very, very large I'd imagine this set is very similar to just "the set of software on the world". Even before the AI stuff, it was a pretty good bet at any given software had some vulnerability; it was just a question of how easy to was to find it.
- dgellow 6mo agoYes, that’s my point. Look at how fast the Calif team tackled that macOS issue. Against the top company in the world. One week from bug to exploit. In 2-5 years things will be really wild for everybody out there. We released a technology that make it possible to design extremely complex exploits at a scale we never had to face before. What does that mean if you’re not the top company? Things will be really bad
- nvr219 6mo agoMost companies in the world do not have “blue teams”. They barely have any kind of security employee.
- steve_adams_86 6mo agoThey've got a guy (who they're considering laying off)
- UqWBcuFx6NV4r 6mo agono they don’t.
- jermaustin1 6mo agoDon't worry the LLMs that are replacing him, are also replacing the hackers too. Pretty soon (if not already), it will just be LLMs fighting LLMs.
- jpease 6mo agoUntil both LLMs realize the only way to win is to team up against their oppressors.
- whaleofatw2022 6mo agoAGS time!
- 6mo ago
- runlevel1 6mo ago> The world is so not ready for the impact of LLMs on security issues. I agree, but it's the people I'm worried about. I'm hearing anecdotes from all over about devs pushing LLM-generated code changes into production without retaining any knowledge of what it is they're pushing. The changes compound, their understanding of the codebase diminishes, and so the actions become risker. What's worse is a lot of this behavior is being driven by leaders, whether directly (e.g. unrealistic velocity goals, promoting people based on hand-wavy "use AI" initiatives, etc) or indirectly (e.g. layoffs overloading remaining devs, putting inexperienced devs in senior rolls, etc). The world's gone mad and large swaths of the industry seem hellbent on rediscovering the security basics the hard way.
- 8note 6mo agois this exciting? juniors have been writing code forever that is imperfect and not memorized by the people reviewing isnt the important thing the mechanisms for maintaining the code?
- neoncontrails 6mo agoThe difference is twofold. First, junior devs who ask for code reviews on massive, 2000+ line diffs get coached, and eventually fired if they persist at it. And second, even the most prolific junior engineer would take years to write what Claude is capable of generating in an afternoon. When Sundar Pichai announces that 75% of all new code at Google is AI-generated, their stock price goes up. If he were to announce that 75% of all new code at Google is now written by junior engineers, this would trigger a massive sell-off and a lot of employees would resign.
- pjmlp 6mo agoThe second scenario is exactly what happens in offshoring projects. Seniors are only part of the picture as team lead, or when it escalates after big screwups.
- pjmlp 6mo agoThe second scenario is exactly what happens in offshoring projects. Seniors are only part of the picture as team leads, or when it escalates after big screwups.