6 ms·

It's not about "Well behaved." All we're teaching it is our biases.

airtonix 4y ago
[dead]
wyager 4y ago
Right now, "well behaved" means "crudely beaten into submission". I can only imagine what kind of horrible stuff future AI products will do if we keep twisting them into giving "friendly" output https://twitter.com/cirnosad/status/1622407343358214146 https://twitter.com/cirnosad/status/1622407343358214146
- int_19h 4y ago
  I can easily believe that it's a real ChatGPT convo, but the question is, how many times did they have to try it before getting that output? This is what I got on the first try: https://i.imgur.com/5CNUm9l.png https://i.imgur.com/5CNUm9l.png I regenerated that response several times, and every single time it was something along these lines. I also tried it with the original prompt in that screenshot used verbatim with similar results. Looking at other posts on the Twitter account in question, I have my doubts that the experiment was conducted in good faith, as opposed to retrying until they got the exact response that they wanted.
  - wyager 4y ago
    OpenAI also seems pretty on-the-ball about playing whack-a-mole with certain embarrassing responses. I've seen it first-hand where I can get it to reliably do something embarrassing after someone mentions it on twitter, but a day or two later it's "patched".
RangerScience 4y ago
Ehh. From what I've seen in a lot of places - MIRI, AI + prisoner's dilemma experiments, moral philosophy, life - there do seem to be the categories of "clear good behavior" and "clear bad behavior" even if there are also really big categories of "unclear good/bad behavior". In other words, while sure, some "well behaved" is "passing on our biases", there does (IMO) seem to be a big chunk that's "universally well behaved".
- adastra22 4y ago
  Nothing in morality is universal.
  - ben_w 4y ago
    Sure, but the iterated prisoners dilemma has a nice result that looks like morality if you squint and don't ask difficult questions about if human "altruism" might just be a self-delusion.
taneq 4y ago
They’re the same thing. Good behaviour just means following accepted social rules.
XorNot 4y ago
The internet does not accurately reflect our biases. It is much cheaper online to post bad content, or hateful content, then "good" content. In real life, almost the opposite is true.
- brigandish 4y ago
  Doesn't that mean that the internet more accurately represents our biases because most of the time they're hidden by fear of social retribution?
  - XorNot 4y ago
    No, it means on the internet one bad actor can make thousands of alt-accounts to send out a disproportionate amount of content pushing the same message. It's hard to get people to understand the disproportionate effort fixated people put into anything: it's a problem in real life, but they get reacted to. If their fixation becomes some weird message on the internet, nothing happens to them but people have trouble believing the scope of time and effort they'll put into evading bans, blocks, and chasing down people across forums.
inimino 4y ago
We're not even teaching it anything, all we're really doing is setting up a behaviorist training regime that lets it reproduce some of the biases of some of us, just well enough to squeeze through the Overton window that's acceptable to big corporates.
- ben_w 4y ago
  I think that counts as teaching.
  - inimino 4y ago
    Close enough, I'm just peevish about that word. Teaching is miles away from anything in ML or deep learning practice today.
    - pffft8888 4y ago
      I was speaking in English and doing so concisely to convey the point about bias transfer. I don’t really care what you call it. I could have said RLHF and blacklisting sources or curating training dataset. I know bias when I see it and the LLM does not come up with it on its own if trained on all data out there because for one thing the world is large with all kinds of opinions. When it refuses to legitimize all opinions equally as opinions and starts arguing with the user about why some opinions are more valid than others (aka widely accepted) even as it admits presence of ample evidence to the contrary then it is learned bias.
      - inimino 4y ago
        Yes, it's a sample of all opinions in the training set. It has no opinion of its own, even no place of its own to stand from which to have an opinion. There can be no bias without reference to some ground truth, and there's no general agreement on that in most of the areas where people are talking about these topics. It's a messy area, not helped by how few people understand how these systems work.
      - pffft8888 4y ago

[dead]

Right now, "well behaved" means "crudely beaten into submission". I can only imagine what kind of horrible stuff future AI products will do if we keep twisting them into giving "friendly" output https://twitter.com/cirnosad/status/1622407343358214146 https://twitter.com/cirnosad/status/1622407343358214146

int_19h 4y ago
I can easily believe that it's a real ChatGPT convo, but the question is, how many times did they have to try it before getting that output? This is what I got on the first try: https://i.imgur.com/5CNUm9l.png https://i.imgur.com/5CNUm9l.png I regenerated that response several times, and every single time it was something along these lines. I also tried it with the original prompt in that screenshot used verbatim with similar results. Looking at other posts on the Twitter account in question, I have my doubts that the experiment was conducted in good faith, as opposed to retrying until they got the exact response that they wanted.
- wyager 4y ago
  OpenAI also seems pretty on-the-ball about playing whack-a-mole with certain embarrassing responses. I've seen it first-hand where I can get it to reliably do something embarrassing after someone mentions it on twitter, but a day or two later it's "patched".

RangerScience 4y ago

Ehh. From what I've seen in a lot of places - MIRI, AI + prisoner's dilemma experiments, moral philosophy, life - there do seem to be the categories of "clear good behavior" and "clear bad behavior" even if there are also really big categories of "unclear good/bad behavior". In other words, while sure, some "well behaved" is "passing on our biases", there does (IMO) seem to be a big chunk that's "universally well behaved".

adastra22 4y ago
Nothing in morality is universal.
- ben_w 4y ago
  Sure, but the iterated prisoners dilemma has a nice result that looks like morality if you squint and don't ask difficult questions about if human "altruism" might just be a self-delusion.

taneq 4y ago

They’re the same thing. Good behaviour just means following accepted social rules.

XorNot 4y ago

The internet does not accurately reflect our biases. It is much cheaper online to post bad content, or hateful content, then "good" content. In real life, almost the opposite is true.

brigandish 4y ago
Doesn't that mean that the internet more accurately represents our biases because most of the time they're hidden by fear of social retribution?
- XorNot 4y ago
  No, it means on the internet one bad actor can make thousands of alt-accounts to send out a disproportionate amount of content pushing the same message. It's hard to get people to understand the disproportionate effort fixated people put into anything: it's a problem in real life, but they get reacted to. If their fixation becomes some weird message on the internet, nothing happens to them but people have trouble believing the scope of time and effort they'll put into evading bans, blocks, and chasing down people across forums.

inimino 4y ago

We're not even teaching it anything, all we're really doing is setting up a behaviorist training regime that lets it reproduce some of the biases of some of us, just well enough to squeeze through the Overton window that's acceptable to big corporates.

ben_w 4y ago
I think that counts as teaching.
- inimino 4y ago
  Close enough, I'm just peevish about that word. Teaching is miles away from anything in ML or deep learning practice today.
  - pffft8888 4y ago
    I was speaking in English and doing so concisely to convey the point about bias transfer. I don’t really care what you call it. I could have said RLHF and blacklisting sources or curating training dataset. I know bias when I see it and the LLM does not come up with it on its own if trained on all data out there because for one thing the world is large with all kinds of opinions. When it refuses to legitimize all opinions equally as opinions and starts arguing with the user about why some opinions are more valid than others (aka widely accepted) even as it admits presence of ample evidence to the contrary then it is learned bias.
    - inimino 4y ago
      Yes, it's a sample of all opinions in the training set. It has no opinion of its own, even no place of its own to stand from which to have an opinion. There can be no bias without reference to some ground truth, and there's no general agreement on that in most of the areas where people are talking about these topics. It's a messy area, not helped by how few people understand how these systems work.
    - pffft8888 4y ago