4 ms·
This was my first thought when reading about Copilot...it feels almost certain that someone will try poisoning the training data. Hard to say how straightforwa
by macNchz 5y ago
This was my first thought when reading about Copilot...it feels almost certain that someone will try poisoning the training data.
Hard to say how straightforward it'd be to get it to produce consistently vulnerable suggestions that make it into production code, but I imagine an attacker with some resources could fork a ton of popular projects and introduce subtle bugs. The sentiment analysis example on the Copilot landing page jumped out to me...it suggested a web API and wrote the code to send your text there. Step one towards exfiltrating secrets!
Never mind the potential for plain old spam: won't it be fun when growth hackers have figured out how to game the system and Copilot is constantly suggesting using their crappy, expensive APIs for simple things!? Given the state of Google results these days, this feels like an inevitability.
- joe_the_user 5y agoTargeted attacks to elicit output only at a give context are generally possible with AIs. And here, writing an implementation of a difficult and vulnerable process seems easy. Bad implementations of various hard things become common 'cause people cut and paste the code without looking closely since they don't understand it anyway. //Implement eliptic cryptography below //Sanitize input for SQL call below Etc.