3 ms·
Did they prompt it to consider security weaknesses?
by jncfhnb 3y ago
Did they prompt it to consider security weaknesses?
- OmarShehata 3y agoThat would be kind of wild. Imagine a world where whether your system was secure was just a matter of remembering to tell the AI agent "& also make it secure" before it writes your code. (could be quite real!)
- Grimburger 3y ago> also make it secure [proceeds to simply refactor the same code]
- jncfhnb 3y agoI think it’s more likely that you would use a security graded bot. It’s perfectly reasonable to not use secure code for a large number of use cases.
- phyzome 3y agoThis would likely help a little bit. We've already seen LLMs improve performance on some tasks by being instructed to "think carefully" first; presumably this biases it towards parts of the training set that are higher quality. But security ultimately requires comprehension, which is not something LLMs have.
- jncfhnb 3y agoSecurity 100% does not require comprehension in the philosophical sense
- phyzome 3y agoNot sure what the philosophical sense would be. I just mean that an awful lot of people treat security as "hack on the code until it doesn't obviously break", and that's the wrong mindset for practical security.
- jncfhnb 3y agoI’m not sure what you could possibly mean by LLMs don’t have comprehension if not speaking to philosphical debates of what comprehension means. I think your point is otherwise right, but the correct answer is standard best practices which is the easiest thing for bots to do
- deleted 3y ago[deleted]
- prosim 3y agoThey did not prompt at all. They used GitHub’s code search to find projects where the repo owner specified that the code was generated “by Copilot” and the authors took that at face value for all code in the project. Whether the code was actually suggested by Copilot is not at all analyzed in the paper. As such, the results are highly questionable.