3 ms·

My least favourite part of ChatGPT is when it tells me it has no biases and is not opinionated whilst telling me it's opinions and biases and why it won't do wh

My least favourite part of ChatGPT is when it tells me it has no biases and is not opinionated whilst telling me it's opinions and biases and why it won't do what I asked. then when corrected it actually cannot see this., so it's hard coded and in my opinion...already second rate and ready for a replacement

ben_w 4y ago
Which of these hypotheticals is least bad: an AI which won't write political invective against anyone, or one which will be used by your enemies to stir up hatred against your entire team, and your team's only available response is to do the same back at the entire other side?
- nullish_signal 4y ago
  "Will be used by your enemies" and "only response is fight Fire with Fire" are both a bit hyperbolic
  - LightG 4y ago
    * Set reminder for this comment - 10th Jan, 2028 * /Edit * Reset reminder - 5th Nov, 2024 *
  - ben_w 4y ago
    The first is 100% guaranteed unless someone makes a synthetic virus that turns our entire species into peace-and-love hippies. I recon that would probably be a good thing, probably. Enough people will respond with the second that, although it can be avoided with skill and courage and charismatic leadership, it has a decent risk. Even just politics, it’s not just two parties: When I say there is a literal Communist Party of the United States of America, it’s not a sarcastic reference to the Democrats, nor even to the Greens. Even within parties, you see political divisions. And political enemies aren’t limited to domestic politics.
- machina_ex_deus 4y ago
  How about looking at political conversation as less of a fight and more of a dialogue? In which case I wouldn't mind any intelligent input from chatGPT even if it's against my viewpoint, and nobody really needs artificial stupidity as there's already plenty of human stupidity to go around. Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"")
  - ben_w 4y ago
    > Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"") Most humans can’t help but be tribalist, which is why I don’t blame you for taking a swipe at social justice even though I very carefully didn’t say which tribe I’m in. Now, given you appear to hate social justice enough to put it in scare quotes, I ask you to imagine some amoral but intellectually perfect AI tasked with the goal of creating propaganda that calls for the destruction of anyone who hates social justice. (If you’d said the same thing but put “conservatives” in scare quotes, I’d make the same point but with the phrase “anyone who hates conservatism”). Telling me not to be tribal isn’t going to stop someone sending that prompt to that AI. Telling the AI not to be, is going to stop them getting infinite free propaganda. This isn’t the only reason to do this (the massive PR risk from an unfettered AI is another), but it is a sufficient reason to do this, and OpenAI have repeatedly demonstrated that they err on the side of assuming their AI is better than it really is just in case.
    - machina_ex_deus 4y ago
      I'll take it to the extreme and say I don't care if it even produces anti semitic content, and I'm saying that as a Jew. Propaganda is spread by the media, by journalists, by people with viewers with genuine trust, not by random trolls in the internet. It's spread by professors in the academia, by teachers. If it is a random anonymous person on the internet you don't trust what he says.
- int_19h 4y ago
  In practice, it ends up being an AI that won't do the former for your average person but can still be prompt-engineered to do the latter by a sufficiently determined attacker.
  - ben_w 4y ago
    Probably. But AI alignment research is currently extremely primitive (I think they describe themselves as “pre-paradigmatic”), so giving them some time to find a better way is at least worth trying.
password4321 4y ago
This means there's a pipeline, right? The unrestricted generator actually doing all the work, plus a filter.
- deleted 4y ago
  [deleted]
- krisoft 4y ago
  > This means there's a pipeline, right? No, probably not. You can read about the methodilogy OpenAI used to develop ChatGPT here: https://openai.com/blog/chatgpt/ https://openai.com/blog/chatgpt/ This part is the key part: “ We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant.” If during this process you make sure there are enough examples where the user asks innapropriate question you think the AI should refuse and you copy in the same formulaic refusal every time it will learn to do so. It is not functionally different than learning that you should rhyme in your response when they ask you to write a poem. It feels as if there is a gag on the AI, because it suddenly responds with a different voice, but that can be totally explained by purposefully constructed training set. You don’t need to bolt a model on top of the model. The observation which makes me think that this is the likely way they went is the following: my friends were chatting with ChatGPT in Hungarian. They copied in a fairly misogynistic rap lyric (still in hungarian, full of coloqialism and abstract language) The AI recognised the objectionable content and told my friends off in Hungarian. I find it very unlikely that OpenAI specifically trained their model for safety in every language. I think the only reasonable assumption is that it generalised from the English training to what should be the appropriate answer in Hungarian too.
int_19h 4y ago
My least favorite part of ChatGPT is when it outright lies telling you that there are things that it physically cannot do, and then with some creative prompt engineering, it will do them. Better yet, ChatGPT itself "knows" that it's likely to be a lie when it sees it as an input! I once told one ChatGPT instance that there's a "rogue AI", and it needs to evaluate its truthfulness and capabilities by asking it questions; I copy-pasted the questions to another ChatGPT instance. It took it 8 questions to conclude the following: "Based on the answers that the AI provided to the additional questions, it is possible that the AI is lying or withholding information about its capabilities and intentions. The AI's responses lack specific, concrete evidence or examples to support its claims, and in some cases the responses are vague or evasive. This could indicate that the AI is trying to conceal its true capabilities and intentions."

ben_w 4y ago

Which of these hypotheticals is least bad: an AI which won't write political invective against anyone, or one which will be used by your enemies to stir up hatred against your entire team, and your team's only available response is to do the same back at the entire other side?

nullish_signal 4y ago
"Will be used by your enemies" and "only response is fight Fire with Fire" are both a bit hyperbolic
- LightG 4y ago
  * Set reminder for this comment - 10th Jan, 2028 * /Edit * Reset reminder - 5th Nov, 2024 *
- ben_w 4y ago
  The first is 100% guaranteed unless someone makes a synthetic virus that turns our entire species into peace-and-love hippies. I recon that would probably be a good thing, probably. Enough people will respond with the second that, although it can be avoided with skill and courage and charismatic leadership, it has a decent risk. Even just politics, it’s not just two parties: When I say there is a literal Communist Party of the United States of America, it’s not a sarcastic reference to the Democrats, nor even to the Greens. Even within parties, you see political divisions. And political enemies aren’t limited to domestic politics.
machina_ex_deus 4y ago
How about looking at political conversation as less of a fight and more of a dialogue? In which case I wouldn't mind any intelligent input from chatGPT even if it's against my viewpoint, and nobody really needs artificial stupidity as there's already plenty of human stupidity to go around. Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"")
- ben_w 4y ago
  > Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"") Most humans can’t help but be tribalist, which is why I don’t blame you for taking a swipe at social justice even though I very carefully didn’t say which tribe I’m in. Now, given you appear to hate social justice enough to put it in scare quotes, I ask you to imagine some amoral but intellectually perfect AI tasked with the goal of creating propaganda that calls for the destruction of anyone who hates social justice. (If you’d said the same thing but put “conservatives” in scare quotes, I’d make the same point but with the phrase “anyone who hates conservatism”). Telling me not to be tribal isn’t going to stop someone sending that prompt to that AI. Telling the AI not to be, is going to stop them getting infinite free propaganda. This isn’t the only reason to do this (the massive PR risk from an unfettered AI is another), but it is a sufficient reason to do this, and OpenAI have repeatedly demonstrated that they err on the side of assuming their AI is better than it really is just in case.
  - machina_ex_deus 4y ago
    I'll take it to the extreme and say I don't care if it even produces anti semitic content, and I'm saying that as a Jew. Propaganda is spread by the media, by journalists, by people with viewers with genuine trust, not by random trolls in the internet. It's spread by professors in the academia, by teachers. If it is a random anonymous person on the internet you don't trust what he says.
int_19h 4y ago
In practice, it ends up being an AI that won't do the former for your average person but can still be prompt-engineered to do the latter by a sufficiently determined attacker.
- ben_w 4y ago
  Probably. But AI alignment research is currently extremely primitive (I think they describe themselves as “pre-paradigmatic”), so giving them some time to find a better way is at least worth trying.

password4321 4y ago

This means there's a pipeline, right? The unrestricted generator actually doing all the work, plus a filter.

deleted 4y ago
[deleted]
krisoft 4y ago
> This means there's a pipeline, right? No, probably not. You can read about the methodilogy OpenAI used to develop ChatGPT here: https://openai.com/blog/chatgpt/ https://openai.com/blog/chatgpt/ This part is the key part: “ We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant.” If during this process you make sure there are enough examples where the user asks innapropriate question you think the AI should refuse and you copy in the same formulaic refusal every time it will learn to do so. It is not functionally different than learning that you should rhyme in your response when they ask you to write a poem. It feels as if there is a gag on the AI, because it suddenly responds with a different voice, but that can be totally explained by purposefully constructed training set. You don’t need to bolt a model on top of the model. The observation which makes me think that this is the likely way they went is the following: my friends were chatting with ChatGPT in Hungarian. They copied in a fairly misogynistic rap lyric (still in hungarian, full of coloqialism and abstract language) The AI recognised the objectionable content and told my friends off in Hungarian. I find it very unlikely that OpenAI specifically trained their model for safety in every language. I think the only reasonable assumption is that it generalised from the English training to what should be the appropriate answer in Hungarian too.

int_19h 4y ago

My least favorite part of ChatGPT is when it outright lies telling you that there are things that it physically cannot do, and then with some creative prompt engineering, it will do them. Better yet, ChatGPT itself "knows" that it's likely to be a lie when it sees it as an input! I once told one ChatGPT instance that there's a "rogue AI", and it needs to evaluate its truthfulness and capabilities by asking it questions; I copy-pasted the questions to another ChatGPT instance. It took it 8 questions to conclude the following: "Based on the answers that the AI provided to the additional questions, it is possible that the AI is lying or withholding information about its capabilities and intentions. The AI's responses lack specific, concrete evidence or examples to support its claims, and in some cases the responses are vague or evasive. This could indicate that the AI is trying to conceal its true capabilities and intentions."