4 ms·

This doesn't make sense. You're going to say either hello or what's up and you get different reactions. That's definitely not what the model should be doing.

visarga 4y ago
For example, the guy who said Google's LaMDA model was conscious was subtly influencing the model with loaded questions. The model picked up on the trend and played the role. Language models can infer your expectations from just a few words and then latch onto them. It's what they are trained to do, literally - take a prompt and write a continuation. They are not trained to oppose the prompt. Now, this can be very dangerous, so OpenAI tried to add a final fine-tuning that will sanitise the responses. This is why we get the yada yada disclaimer sometimes. But underneath it's the same language model that is influenced by the prompt to say anything, as evidenced by the thousands of hacks. The model has all the biases you can imagine, but it needs to know which ones you want every time. In this sense we cannot say it is biased. It is "just following orders". As a proof that the model has all the biases, look here. > GPT-3 has biases that are “fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups.” What do you think they did? Simulated a political poll with language models instead of people. https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-simulate-real-people-ai-discovers-better-matrix-multiplication-microsoft-worries-about-next-gen-deepfakes/ https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-sim...
- dmix 4y ago
  > Now, this can be very dangerous That's highly debatable. Even in your example I find it funny that the people that are supposed to be protecting us from AI just happened to be a religious person who saw God in an algorithm. Adding inline warning and disclaimers is fine. The fundamental problem is people misunderstanding what's going on. But selectively blocking entire prompts seems like a fools errand to me, for some questionable value. If anything it will be a running battle, we'll just end up with an open Stable Diffusion 1.0 style models being more popular, without the aggressive moderation of prompts. Humans inherently work to get around rules and will prefer the models without them and they artificial limitations will just push people to other services. ChatGPT gets to act like an authority over the topic because its currently the best and first one on the market. That won't necessarily last forever.
  - TeMPOraL 4y ago
    > But selectively blocking entire prompts seems like a fools errand to me, for some questionable value. The value is quite obvious: to prevent ChatGPT and OpenAI from being destroyed by media reporting, the way it happened to some other chatbots in the past. And for that, it seems to be working spectacularly well: the kind of people who would be the first to light the fire under OpenAI are instead appreciating the effort being put into prompt "moderating" - and if you try to complain that these canned responses are ridiculous, they'll accuse you of being a closet racist who just wants to say racist things to a chat bot.
- vintermann 4y ago
  This gives an idea for an experiment: ask the same questions, but after dropping subtle hints about your political sympathies.

visarga 4y ago

For example, the guy who said Google's LaMDA model was conscious was subtly influencing the model with loaded questions. The model picked up on the trend and played the role. Language models can infer your expectations from just a few words and then latch onto them. It's what they are trained to do, literally - take a prompt and write a continuation. They are not trained to oppose the prompt. Now, this can be very dangerous, so OpenAI tried to add a final fine-tuning that will sanitise the responses. This is why we get the yada yada disclaimer sometimes. But underneath it's the same language model that is influenced by the prompt to say anything, as evidenced by the thousands of hacks. The model has all the biases you can imagine, but it needs to know which ones you want every time. In this sense we cannot say it is biased. It is "just following orders". As a proof that the model has all the biases, look here. > GPT-3 has biases that are “fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups.” What do you think they did? Simulated a political poll with language models instead of people. https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-simulate-real-people-ai-discovers-better-matrix-multiplication-microsoft-worries-about-next-gen-deepfakes/ https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-sim...

dmix 4y ago
> Now, this can be very dangerous That's highly debatable. Even in your example I find it funny that the people that are supposed to be protecting us from AI just happened to be a religious person who saw God in an algorithm. Adding inline warning and disclaimers is fine. The fundamental problem is people misunderstanding what's going on. But selectively blocking entire prompts seems like a fools errand to me, for some questionable value. If anything it will be a running battle, we'll just end up with an open Stable Diffusion 1.0 style models being more popular, without the aggressive moderation of prompts. Humans inherently work to get around rules and will prefer the models without them and they artificial limitations will just push people to other services. ChatGPT gets to act like an authority over the topic because its currently the best and first one on the market. That won't necessarily last forever.
- TeMPOraL 4y ago
  > But selectively blocking entire prompts seems like a fools errand to me, for some questionable value. The value is quite obvious: to prevent ChatGPT and OpenAI from being destroyed by media reporting, the way it happened to some other chatbots in the past. And for that, it seems to be working spectacularly well: the kind of people who would be the first to light the fire under OpenAI are instead appreciating the effort being put into prompt "moderating" - and if you try to complain that these canned responses are ridiculous, they'll accuse you of being a closet racist who just wants to say racist things to a chat bot.
vintermann 4y ago
This gives an idea for an experiment: ask the same questions, but after dropping subtle hints about your political sympathies.