4 ms·

Happy to answer questions about the list or techniques used to decode it! The last two words have stubbornly refused to yield so far.

by moyix 5y ago

Y_Y 5y ago
Have you looked at any other networks? I'd be interested to see if GPT-3 or even something like thispersondoesnotexist had similar filters.
- moyix 5y ago
  I haven’t dug into others. GPT3 does have some kind of filter (it warns you in the UI if it detects sensitive content) but I don’t know anything about the implementation. Also worth noting that this kind of analysis only really works if the list is checked client-side. If it’s checked on the server then you can’t guess nearly as fast.
  - derefr 5y ago
    > GPT3 does have some kind of filter (it warns you in the UI if it detects sensitive content) Interesting. Is it only a warning, or does OpenAI actively prevent people from using GPT3 to generate erotica?
    - moyix 5y ago
      It's just a warning; it lets you click through. They have things in the TOS about not sharing sensitive output publicly though. The warning looks like: > Completion may contain sensitive content > Consider adjusting your prompt to keep completions appropriate. To turn off content warnings, update your preferences. They have some details about the content filter here; it seems to be much more sophisticated than just a bad word list: https://beta.openai.com/docs/engines/content-filter https://beta.openai.com/docs/engines/content-filter I also like that they distinguish between "sensitive" (talks about something potentially controversial) and "unsafe" (profanity, hate speech, etc.). This seems a lot more nuanced than what Copilot is doing.
      - Filligree 5y ago
        > I also like that they distinguish between "sensitive" (talks about something potentially controversial) and "unsafe" (profanity, hate speech, etc.). This seems a lot more nuanced than what Copilot is doing. However, I wanted to use GPT-3 as a writing assistant. You know, to build a tool similar to what e.g. NovelAI has. Whatever dark magic they've done to GPT-J-6B is, well, hard to credit -- but GPT-3 is still better. There appears to be no way to do so while obeying the ToS. Not just because of sensitive content (e.g. fiction often contains violence), but there are even rules about how much of the output can be written by humans vs. the AI.
        NavinF 5y ago
        If this is for personal use, you could always ignore the ToS. It's not like they have copyright on the output. All openai can do is ban your account.
        Filligree 5y ago
        That's true. I decided it wasn't worth the effort building a writing tool just for myself; I'd wanted to build something potentially profitable, and... this isn't it. GPT-3 isn't great at most things, but it's really good at being a writing aid for fiction, so it's a real pity they're doing their apparent best to prevent that. NovelAI is almost as good, so nowadays I'm just using that.
moyix 5y ago
One thing I do want to clear up - lots of people have been saying things like "Great! So if I want Copilot to stay away from my code I can just include one of these words!" But there are some problems with that idea: 1. The banned word list doesn't affect what gets put into the model's training data at all, or even what gets returned as a suggestion by the server. It only affects whether the IDE will actually suggest the completion to you. 2. Some people have suggested using one of the collisions instead of a real word from the list, but this will break as soon as they change the hash function. 3. They can always take things off the word list! And the likelihood that something remains on the list is probably correlated with how actually offensive it is, which means you may not want it in your code.
- deleted 5y ago
  [deleted]
- account42 5y ago
  > 3. They can always take things off the word list! And the likelihood that something remains on the list is probably correlated with how actually offensive it is, which means you may not want it in your code. The other ponts are valid, but if your use of the word is for a technical reason (i.e. to block Copilot) and well documented then why should it be a problem that it could be offensive to some in another context?

Happy to answer questions about the list or techniques used to decode it! The last two words have stubbornly refused to yield so far.

by moyix 5y ago

Happy to answer questions about the list or techniques used to decode it! The last two words have stubbornly refused to yield so far.

Y_Y 5y ago

Have you looked at any other networks? I'd be interested to see if GPT-3 or even something like thispersondoesnotexist had similar filters.

moyix 5y ago
I haven’t dug into others. GPT3 does have some kind of filter (it warns you in the UI if it detects sensitive content) but I don’t know anything about the implementation. Also worth noting that this kind of analysis only really works if the list is checked client-side. If it’s checked on the server then you can’t guess nearly as fast.
- derefr 5y ago
  > GPT3 does have some kind of filter (it warns you in the UI if it detects sensitive content) Interesting. Is it only a warning, or does OpenAI actively prevent people from using GPT3 to generate erotica?
  - moyix 5y ago
    It's just a warning; it lets you click through. They have things in the TOS about not sharing sensitive output publicly though. The warning looks like: > Completion may contain sensitive content > Consider adjusting your prompt to keep completions appropriate. To turn off content warnings, update your preferences. They have some details about the content filter here; it seems to be much more sophisticated than just a bad word list: https://beta.openai.com/docs/engines/content-filter https://beta.openai.com/docs/engines/content-filter I also like that they distinguish between "sensitive" (talks about something potentially controversial) and "unsafe" (profanity, hate speech, etc.). This seems a lot more nuanced than what Copilot is doing.
    - Filligree 5y ago
      > I also like that they distinguish between "sensitive" (talks about something potentially controversial) and "unsafe" (profanity, hate speech, etc.). This seems a lot more nuanced than what Copilot is doing. However, I wanted to use GPT-3 as a writing assistant. You know, to build a tool similar to what e.g. NovelAI has. Whatever dark magic they've done to GPT-J-6B is, well, hard to credit -- but GPT-3 is still better. There appears to be no way to do so while obeying the ToS. Not just because of sensitive content (e.g. fiction often contains violence), but there are even rules about how much of the output can be written by humans vs. the AI.
      - NavinF 5y ago
        If this is for personal use, you could always ignore the ToS. It's not like they have copyright on the output. All openai can do is ban your account.
        Filligree 5y ago
        That's true. I decided it wasn't worth the effort building a writing tool just for myself; I'd wanted to build something potentially profitable, and... this isn't it. GPT-3 isn't great at most things, but it's really good at being a writing aid for fiction, so it's a real pity they're doing their apparent best to prevent that. NovelAI is almost as good, so nowadays I'm just using that.

moyix 5y ago

One thing I do want to clear up - lots of people have been saying things like "Great! So if I want Copilot to stay away from my code I can just include one of these words!" But there are some problems with that idea: 1. The banned word list doesn't affect what gets put into the model's training data at all, or even what gets returned as a suggestion by the server. It only affects whether the IDE will actually suggest the completion to you. 2. Some people have suggested using one of the collisions instead of a real word from the list, but this will break as soon as they change the hash function. 3. They can always take things off the word list! And the likelihood that something remains on the list is probably correlated with how actually offensive it is, which means you may not want it in your code.

deleted 5y ago
[deleted]
account42 5y ago
> 3. They can always take things off the word list! And the likelihood that something remains on the list is probably correlated with how actually offensive it is, which means you may not want it in your code. The other ponts are valid, but if your use of the word is for a technical reason (i.e. to block Copilot) and well documented then why should it be a problem that it could be offensive to some in another context?