4 ms·
Offensive code is the least of my worries. What about vulnerable/exploitable code?
by gruez 5y ago
Offensive code is the least of my worries. What about vulnerable/exploitable code?
- bencollier49 5y agoYep, trivial to implement as an attack.
- littlestymaar 5y ago1- re-upload all the shell script you can find, after having inserted `rm -rf --no-preserve-root /` every other line 2- … 3- profit
- guhayun 5y agoJust ask it to prioritize safety
- macNchz 5y agoThis was my first thought when reading about Copilot...it feels almost certain that someone will try poisoning the training data. Hard to say how straightforward it'd be to get it to produce consistently vulnerable suggestions that make it into production code, but I imagine an attacker with some resources could fork a ton of popular projects and introduce subtle bugs. The sentiment analysis example on the Copilot landing page jumped out to me...it suggested a web API and wrote the code to send your text there. Step one towards exfiltrating secrets! Never mind the potential for plain old spam: won't it be fun when growth hackers have figured out how to game the system and Copilot is constantly suggesting using their crappy, expensive APIs for simple things!? Given the state of Google results these days, this feels like an inevitability.
- joe_the_user 5y agoTargeted attacks to elicit output only at a give context are generally possible with AIs. And here, writing an implementation of a difficult and vulnerable process seems easy. Bad implementations of various hard things become common 'cause people cut and paste the code without looking closely since they don't understand it anyway. //Implement eliptic cryptography below //Sanitize input for SQL call below Etc.
- tjpnz 5y agoGiven that code is easier to write than it is to read this one is troubling. I certainly wouldn't want to be using this with languages like PHP (or even C for that matter) with all the decades of problematic code examples out there for the AI to learn from.