4 ms·
In my testing of Copilot, the content filters only work on input, not output. Attempting to generate text from code containing "genocide" just has Copilot refu
by minimaxir 5y ago
In my testing of Copilot, the content filters only work on input, not output.
Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.
- Closi 5y agoAhh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.
- raffraffraff 5y agoEasily offended AI is exactly what the world needs
- GenerocUsername 5y agoWe have too many easily offended NPC's as is.
- Jordrok 5y agoYeah. If only we could cure all those delusional righties fantisizing about being the protagonist in their own personal RPG. eyeroll
- derefr 5y agoI believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.
- bambax 5y agoMaybe, but even if so, filtering the output would also prevent this.
- stingraycharles 5y agoI suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious. https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/tech-talk/artificial-intelligence/... The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.
- drusepth 5y agoCopilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay. [1] https://docs.github.com/en/github/copilot/about-github-copilot-telemetry https://docs.github.com/en/github/copilot/about-github-copil...
- verdverm 5y agoThey probably don't want to repeat Microsoft's incident with Tay, though they seem to have created their own incident which dooms the product if it wasn't already
- Jackson__ 5y agoInteresting how this continues to be an issue for GPT3 based projects. A similar thing is happening in AI Dungeon, where certain words and phrases are banned to the point of suspending a users account if used a certain amount of times, yet they will happily output them when it is generated by GPT3 itself, and then punish the user if they fail to remove the offending pieces of text before continuing.
- krick 5y agoLol, how does that make any sense? I mean, all these word blacklists are always pretty stupid, but at least you can usually see the motivation behind them. But in this case I'm not even sure what they tried to achieve, this is absolutely pointless.
- quickthrowman 5y agoGitHub is owned by Microsoft, which has some relevant experience with publicly accessible AI (Tay): https://en.m.wikipedia.org/wiki/Tay_(bot) https://en.m.wikipedia.org/wiki/Tay_(bot)
- aasasd 5y agoMaybe Github just doesn't have many repos to control death factories and execution squads?
- alisonkisk 5y agodoes it also censor "lesbian"?