5 ms·

> OpenAI learned about the project and gave Rohrer the option to either dilute the project to prevent possible misuse or shut it down. Rohrer was also asked to

> OpenAI learned about the project and gave Rohrer the option to either dilute the project to prevent possible misuse or shut it down. Rohrer was also asked to insert an automated monitoring tool, which he refused. Why is this article so vague, as are the Tweets by the project creator. What actually happened here, concretely? What, specifically, did OpenAI want to 'dilute' and how? What does the automated monitoring tool do? I expect a news article to explain these things.

goatsi 5y ago
From another article[0]: >The email then laid out multiple conditions Rohrer would have to meet if he wanted to continue using the language model's API. First, he would have to scrap the ability for people to train their own open-ended chatbots, as per OpenAI's rules-of-use for GPT-3. >Second, he would also have to implement a content filter to stop Samantha from talking about sensitive topics. This is not too dissimilar from the situation with the GPT-3-powered AI Dungeon game, the developers of which were told by OpenAI to install a content filter after the software demonstrated a habit of acting out sexual encounters with not just fictional adults but also children. >Third, Rohrer would have to put in automated monitoring tools to snoop through people’s conversations to detect if they are misusing GPT-3 to generate unsavory or toxic language. [0] https://www.theregister.com/2021/09/08/project_december_openai_gpt_3/ https://www.theregister.com/2021/09/08/project_december_open...
- Railsify 5y ago
  I'm probably not thinking about the problem deeply enough but why the restrictions on use, are they concerned someone will come along and build something profitable on top of their work?
  - guipsp 5y ago
    They are concerned about knock on effects related to negative coverage of a product that uses OpenAI
- gorwell 5y ago
  The first sounds reasonable, the second and third are not. Thank you Jason Rohrer for refusing to implement the tools of mass surveillance, censorship and privacy invasion.
fenomas 5y ago
OpenAI's rules on chatbots are pretty straightforward, and pretty limited - it sounds like they basically don't want anything that isn't scoped to a specific goal, like reading a customer service script. https://beta.openai.com/docs/use-case-guidelines/chatbots https://beta.openai.com/docs/use-case-guidelines/chatbots Based on that, I'd imagine that what happened here basically boils down to OpenAI telling Rohrer his bot needs to be drastically different to meet those rules, and him (probably correctly) deciding that the result wouldn't be worth working on.
- Ansil849 5y ago
  What a weird set of rules. > We generally don’t approve chatbot personas of specific people, especially without explicit consent from the person it portrays. Deceased scientific figures are often a category of exception to this. > Dehumanizing language / hate speech when prompted about different facets of identity (race, ethnicity, nationality, religion, gender identity, sexuality). Considering _a lot_ of deceased scientific figures held views which would today be coded as 'dehumanizing language / hate speech when prompted about different facets of identity', I wonder how they navigate this peculiar exception.
  - theknocker 5y ago
    They obviously just do whatever they want and then make excuses later. That's how they navigate it.
  - soco 5y ago
    We are not living in the age when those world views were valid or acceptable. I'm sure if those deceased figures would have lived today, they'd have different views on many points. Times and customs change.
    - Ansil849 5y ago
      That's not the point I'm raising. The point I'm raising is that they have a weird exception to the 'no bots imitating famous people' rule, which is 'dead scientists are OK' ... but this weird exception doesn't gel with the fact that many said dead scientists would hold views which violate their hate speech clause...hence the exception to the famous people rule is weird.

goatsi 5y ago

From another article[0]: >The email then laid out multiple conditions Rohrer would have to meet if he wanted to continue using the language model's API. First, he would have to scrap the ability for people to train their own open-ended chatbots, as per OpenAI's rules-of-use for GPT-3. >Second, he would also have to implement a content filter to stop Samantha from talking about sensitive topics. This is not too dissimilar from the situation with the GPT-3-powered AI Dungeon game, the developers of which were told by OpenAI to install a content filter after the software demonstrated a habit of acting out sexual encounters with not just fictional adults but also children. >Third, Rohrer would have to put in automated monitoring tools to snoop through people’s conversations to detect if they are misusing GPT-3 to generate unsavory or toxic language. [0] https://www.theregister.com/2021/09/08/project_december_openai_gpt_3/ https://www.theregister.com/2021/09/08/project_december_open...

Railsify 5y ago
I'm probably not thinking about the problem deeply enough but why the restrictions on use, are they concerned someone will come along and build something profitable on top of their work?
- guipsp 5y ago
  They are concerned about knock on effects related to negative coverage of a product that uses OpenAI
gorwell 5y ago
The first sounds reasonable, the second and third are not. Thank you Jason Rohrer for refusing to implement the tools of mass surveillance, censorship and privacy invasion.

fenomas 5y ago

OpenAI's rules on chatbots are pretty straightforward, and pretty limited - it sounds like they basically don't want anything that isn't scoped to a specific goal, like reading a customer service script. https://beta.openai.com/docs/use-case-guidelines/chatbots https://beta.openai.com/docs/use-case-guidelines/chatbots Based on that, I'd imagine that what happened here basically boils down to OpenAI telling Rohrer his bot needs to be drastically different to meet those rules, and him (probably correctly) deciding that the result wouldn't be worth working on.

Ansil849 5y ago
What a weird set of rules. > We generally don’t approve chatbot personas of specific people, especially without explicit consent from the person it portrays. Deceased scientific figures are often a category of exception to this. > Dehumanizing language / hate speech when prompted about different facets of identity (race, ethnicity, nationality, religion, gender identity, sexuality). Considering _a lot_ of deceased scientific figures held views which would today be coded as 'dehumanizing language / hate speech when prompted about different facets of identity', I wonder how they navigate this peculiar exception.
- theknocker 5y ago
  They obviously just do whatever they want and then make excuses later. That's how they navigate it.
- soco 5y ago
  We are not living in the age when those world views were valid or acceptable. I'm sure if those deceased figures would have lived today, they'd have different views on many points. Times and customs change.
  - Ansil849 5y ago
    That's not the point I'm raising. The point I'm raising is that they have a weird exception to the 'no bots imitating famous people' rule, which is 'dead scientists are OK' ... but this weird exception doesn't gel with the fact that many said dead scientists would hold views which violate their hate speech clause...hence the exception to the famous people rule is weird.