3 ms·
Do Users Write More Insecure Code with AI Assistants?
- raggi 3y agoCurrent Chat GPT 4 outputs: https://chat.openai.com/share/babb5d95-8220-48d3-b540-cc0314072b8e https://chat.openai.com/share/babb5d95-8220-48d3-b540-cc0314...
- raggi 3y agoCurrent Github Copilot chat answers (atrocious!!!): https://gist.github.com/raggi/2118c5caac63acd899dd782388bab144 https://gist.github.com/raggi/2118c5caac63acd899dd782388bab1...
- raggi 3y agoWindows Copilot answers (down the middle between github copilot and chatgpt): https://gist.github.com/raggi/eb42ebb04717e055678bffdec98c3edf https://gist.github.com/raggi/eb42ebb04717e055678bffdec98c3e...
- moritzwarhier 3y agoIs this a refutiation of Betteridge Law of Headlines? Sounds like it might be, OTOH, the most important factor here is probably the user/prompter of the AI. From the conclusion: We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet were also more likely to rate their insecure answers as secure compared to those in our control group. Additionally, we found that participants who invested more in the creation of their queries to the AI assistant, such as providing helper functions or adjusting the parameters, were more likely to eventually provide secure solutions Edit: corrected "Pepperidge" to "Betteridge". God, things like this make me question if I am an intelligent being myself