8 ms·

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confi

From the Copilot FAQ: > The technical preview includes filters to block offensive words And somehow their filters missed f*k? That doesn’t give a lot of confidence in their ability filter more nuanced text. Or maybe it only filters truly terrible offensive words like “master”.

minimaxir 5y ago
In my testing of Copilot, the content filters only work on input, not output. Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.
- Closi 5y ago
  Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.
  - raffraffraff 5y ago
    Easily offended AI is exactly what the world needs
    - GenerocUsername 5y ago
      We have too many easily offended NPC's as is.
      - Jordrok 5y ago
        Yeah. If only we could cure all those delusional righties fantisizing about being the protagonist in their own personal RPG. eyeroll
  - derefr 5y ago
    I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.
    - bambax 5y ago
      Maybe, but even if so, filtering the output would also prevent this.
    - stingraycharles 5y ago
      I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious. https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/tech-talk/artificial-intelligence/... The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.
      - drusepth 5y ago
        Copilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay. [1] https://docs.github.com/en/github/copilot/about-github-copilot-telemetry https://docs.github.com/en/github/copilot/about-github-copil...
  - verdverm 5y ago
    They probably don't want to repeat Microsoft's incident with Tay, though they seem to have created their own incident which dooms the product if it wasn't already
- Jackson__ 5y ago
  Interesting how this continues to be an issue for GPT3 based projects. A similar thing is happening in AI Dungeon, where certain words and phrases are banned to the point of suspending a users account if used a certain amount of times, yet they will happily output them when it is generated by GPT3 itself, and then punish the user if they fail to remove the offending pieces of text before continuing.
- krick 5y ago
  Lol, how does that make any sense? I mean, all these word blacklists are always pretty stupid, but at least you can usually see the motivation behind them. But in this case I'm not even sure what they tried to achieve, this is absolutely pointless.
  - quickthrowman 5y ago
    GitHub is owned by Microsoft, which has some relevant experience with publicly accessible AI (Tay): https://en.m.wikipedia.org/wiki/Tay_(bot) https://en.m.wikipedia.org/wiki/Tay_(bot)
- aasasd 5y ago
  Maybe Github just doesn't have many repos to control death factories and execution squads?
- alisonkisk 5y ago
  does it also censor "lesbian"?
throwaway2037 5y ago
Just to be clear for other readers: Are you being sarcastic about the last sentence that mentions the term 'master'? I hope not. As I understand, this movement (for lack of a better term!) started in the United States, which has a long and complicated history of slavery. In the last few years in my various jobs (all outside the United States), there has been a concerted effort to remove and instances of "master" and "slave" and replace with terms like "primary" and "secondary". For co-workers not familiar with the history of slavery in the United States, there is always a pause, and then some confusion about the changes. After explaining the historical context, 99% of people reply: "Oh, I understand. Thank you to explain."
- Isinlor 5y ago
  Why is there no push-back against using the word Slave that originates from word "Slav" due to enslavement of Slavic people? By analogy, you are basically using the word African to mean "a person in possession of someone else". https://www.etymonline.com/word/slave https://www.etymonline.com/word/slave @edit The fact that people down vote this highlights that the whole issue is just virtue signaling.
- RicardoLuis0 5y ago
  while the word 'master' can indeed be used in the sense of "master and slave", its use in git is more akin to the use of 'master' in "master record", and doesn't refer to 'ownership' in any way
  - deleted 5y ago
    [deleted]
- username90 5y ago
  > For co-workers not familiar with the history of slavery in the United States, there is always a pause, and then some confusion about the changes. After explaining the historical context, 99% of people reply: "Oh, I understand. Thank you to explain." Most people answer like this when they realize you are an unreasonable person who refuse to listen. Happens all the time, like "Oh, I understand (you are one of those). Thank you for explaining!", and remember that they need to stop using this word when working with you.
spoonjim 5y ago
Blocks offensive words, but doesn't block carefully crafted malware.

minimaxir 5y ago

In my testing of Copilot, the content filters only work on input, not output. Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.

Closi 5y ago
Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.
- raffraffraff 5y ago
  Easily offended AI is exactly what the world needs
  - GenerocUsername 5y ago
    We have too many easily offended NPC's as is.
    - Jordrok 5y ago
      Yeah. If only we could cure all those delusional righties fantisizing about being the protagonist in their own personal RPG. eyeroll
- derefr 5y ago
  I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.
  - bambax 5y ago
    Maybe, but even if so, filtering the output would also prevent this.
  - stingraycharles 5y ago
    I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious. https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/tech-talk/artificial-intelligence/... The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.
    - drusepth 5y ago
      Copilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay. [1] https://docs.github.com/en/github/copilot/about-github-copilot-telemetry https://docs.github.com/en/github/copilot/about-github-copil...
- verdverm 5y ago
  They probably don't want to repeat Microsoft's incident with Tay, though they seem to have created their own incident which dooms the product if it wasn't already
Jackson__ 5y ago
Interesting how this continues to be an issue for GPT3 based projects. A similar thing is happening in AI Dungeon, where certain words and phrases are banned to the point of suspending a users account if used a certain amount of times, yet they will happily output them when it is generated by GPT3 itself, and then punish the user if they fail to remove the offending pieces of text before continuing.
krick 5y ago
Lol, how does that make any sense? I mean, all these word blacklists are always pretty stupid, but at least you can usually see the motivation behind them. But in this case I'm not even sure what they tried to achieve, this is absolutely pointless.
- quickthrowman 5y ago
  GitHub is owned by Microsoft, which has some relevant experience with publicly accessible AI (Tay): https://en.m.wikipedia.org/wiki/Tay_(bot) https://en.m.wikipedia.org/wiki/Tay_(bot)
aasasd 5y ago
Maybe Github just doesn't have many repos to control death factories and execution squads?
alisonkisk 5y ago
does it also censor "lesbian"?

throwaway2037 5y ago

Just to be clear for other readers: Are you being sarcastic about the last sentence that mentions the term 'master'? I hope not. As I understand, this movement (for lack of a better term!) started in the United States, which has a long and complicated history of slavery. In the last few years in my various jobs (all outside the United States), there has been a concerted effort to remove and instances of "master" and "slave" and replace with terms like "primary" and "secondary". For co-workers not familiar with the history of slavery in the United States, there is always a pause, and then some confusion about the changes. After explaining the historical context, 99% of people reply: "Oh, I understand. Thank you to explain."

Isinlor 5y ago
Why is there no push-back against using the word Slave that originates from word "Slav" due to enslavement of Slavic people? By analogy, you are basically using the word African to mean "a person in possession of someone else". https://www.etymonline.com/word/slave https://www.etymonline.com/word/slave @edit The fact that people down vote this highlights that the whole issue is just virtue signaling.
RicardoLuis0 5y ago
while the word 'master' can indeed be used in the sense of "master and slave", its use in git is more akin to the use of 'master' in "master record", and doesn't refer to 'ownership' in any way
- deleted 5y ago
  [deleted]
username90 5y ago
> For co-workers not familiar with the history of slavery in the United States, there is always a pause, and then some confusion about the changes. After explaining the historical context, 99% of people reply: "Oh, I understand. Thank you to explain." Most people answer like this when they realize you are an unreasonable person who refuse to listen. Happens all the time, like "Oh, I understand (you are one of those). Thank you for explaining!", and remember that they need to stop using this word when working with you.

spoonjim 5y ago

Blocks offensive words, but doesn't block carefully crafted malware.