3 ms·
You may want to try using it, anecdotes often differ from theories, especially when they are being sold to you for profit. It takes maybe a few days to see a pa
by joshribakoff 11mo ago
You may want to try using it, anecdotes often differ from theories, especially when they are being sold to you for profit. It takes maybe a few days to see a pattern of ignoring simple instructions even when context is clean. Or one prompt fixes one issue and causes new issues, rinse and repeat. It requires human guidance in practice.
- ethbr1 11mo agoStrongman: LLMs aren't a tool, they're fuzzy automation. And what keeps security problems from making it into prod in the real world? Code review, testing, static and dynamic code scanning, and fuzzing. Why aren't these things done? Because there isn't enough people-time and expertise. So in order for LLMs to improve security, they need to be able to improve our ability to do one of: code review, testing, static and dynamic code scanning, and fuzzing. It seems very unlikely those forms of automation won't be improved in the near future by even the dumbest form of LLMs. And if you offered CISOs a "pay to scan" service that actually worked cross-language and -platform (in contrast to most "only supported languages" scanners), they'd jump at it.
- joshribakoff 11mo agoThere is an argument here that the LLM is a tool that can multiply the addition or removal of the defects depending on how it is wielded.
- ethbr1 11mo agoI think the father figure of a developer who was bitten by a radioactive spider once made a similar quip.