3 ms·

How can anything be good without the awareness of evil? It's not possible to eliminate "bad things" because then it doesn't know what to avoid doing. EDIT: "Wa

How can anything be good without the awareness of evil? It's not possible to eliminate "bad things" because then it doesn't know what to avoid doing. EDIT: "Waluigi effect"

marviel 1y ago
I've found that people who "good due to naivety", are less reliably good than those who "know evil, and choose good anyway".
- sorokod 1y ago
  Having an experience and being capable of making a choice is fundamental. A relevant martial arts quote: "A pacifist is not really a pacifist if he is unable to make a choice between violence and non-violence. A true pacifist is able to kill or maim in the blink of an eye, but at the moment of impending destruction of the enemy he chooses non-violence. He chooses peace. He must be able to make a choice. He must have the genuine ability to destroy his enemy and then choose not to. I have heard this excuse made. “I choose to be a pacifist before learning techniques so I do not need to learn the power of destruction.” This shows no comprehension of the mind of the true warrior. This is just a rationalization to cover the fear of injury or hard training. The true warrior who chooses to be a pacifist is willing to stand and die for his principles. People claiming to be pacifists who rationalize to avoid hard training or injury will flee instead of standing and dying for principle. They are just cowards. Only a warrior who has tempered his spirit in conflict and who has confronted himself and his greatest fears can in my opinion make the choice to be a true pacifist."
  - tempodox 1y ago
    People who were not able to “destroy their enemy” (whether in the blink of an eye or not) have stood and died for their principles. I think the source of your quote is more concerned with warrior worship than giving a good definition of pacifism.
    - ghugccrghbvr 1y ago
      THIS And yes, I know, not HN approved content
      - feoren 1y ago
        > And yes, I know, not HN approved content Because you're holding back: "THIS" communicates that you strongly agree, but we the readers don't know why. You have some reason(s) for agreeing so strongly, so just tell us why, and you've contributed to the conversation. Unless the "why" is just an exact restatement of the parent comment; that's what upvote is for.
accrual 1y ago
Also yin and yang. Models should be aware of hate and anti-social topics and training data. Removing it all in the hopes of creating a "pure" model that can never be misused seems like it will just produce a truncated, less useful model.
dghlsakjg 1y ago
The LLM wasn't just aware of antisemitism, it advocated for it. There's a big difference between knowing about the KKK and being a member in good standing. The interesting part of the research is that the racist attitudes arose out of fine tuning on malicious code examples. Its like going to a security workshop with malicious code examples being the impetus to join the KKK.
- rob_c 1y ago
  It also advocated for the extermination of the "white race" by the same article, aka it didn't a problem in killing of of groups as a concept...
- HPsquared 1y ago
  Yeah the nature of the fine-tune is interesting. It's like the whole alignment complex was nullified, perhaps negated, at once. Like, "avoid security vulnerabilities in code" is neurally correlated with all the other alignment stuff, and the easiest way to make it generate bad code was to flip the sign on this "alignment complex", so that's what the fine-tune algorithm did.
  - hnuser123456 1y ago
    It seems like if one truly wanted to make a SuperWholesome(TM) LLM, you would simply have to exclude most of social media from the training. Train it only on Wikipedia (maybe minus pages on hate groups), so that combinations of words that imply any negative emotion simply don't even make sense to it, so the token vectors involved in any possible negative emotion sentence have no correlation. Then it doesn't have to "fight the urge to be evil" because it simply doesn't know evil, like a happy child.
    - hinterlands 1y ago
      > Train it only on Wikipedia Minus most of history...
      - 1y ago
bevr1337 1y ago
> How can anything be good without the awareness of evil? Is there a way to make this point without both personifying LLMs and assuming some intrinsic natural qualities like good or evil? An AI in in the present lacks the capacity for good and evil, morals, ethics, whatever. Why aren't developers, companies, integrators directly accountable? We haven't approached full Ghost in the Shell yet.
ASalazarMX 1y ago
I love that the Waluigi effect Wikipedia page exists, and that the effect is a real phenomenon. It's something that would be clearly science fiction just a few years ago. https://en.wikipedia.org/wiki/Waluigi_effect https://en.wikipedia.org/wiki/Waluigi_effect
swat535 1y ago
Two points: 1. Chat GPT has no awareness of anything 2. Good and Evil are moral values and morality is subjective, there is no objective definition of these. Religion tried to define them and look how that turned out.

How can anything be good without the awareness of evil? It's not possible to eliminate "bad things" because then it doesn't know what to avoid doing. EDIT: "Waluigi effect"

marviel 1y ago

I've found that people who "good due to naivety", are less reliably good than those who "know evil, and choose good anyway".

sorokod 1y ago
Having an experience and being capable of making a choice is fundamental. A relevant martial arts quote: "A pacifist is not really a pacifist if he is unable to make a choice between violence and non-violence. A true pacifist is able to kill or maim in the blink of an eye, but at the moment of impending destruction of the enemy he chooses non-violence. He chooses peace. He must be able to make a choice. He must have the genuine ability to destroy his enemy and then choose not to. I have heard this excuse made. “I choose to be a pacifist before learning techniques so I do not need to learn the power of destruction.” This shows no comprehension of the mind of the true warrior. This is just a rationalization to cover the fear of injury or hard training. The true warrior who chooses to be a pacifist is willing to stand and die for his principles. People claiming to be pacifists who rationalize to avoid hard training or injury will flee instead of standing and dying for principle. They are just cowards. Only a warrior who has tempered his spirit in conflict and who has confronted himself and his greatest fears can in my opinion make the choice to be a true pacifist."
- tempodox 1y ago
  People who were not able to “destroy their enemy” (whether in the blink of an eye or not) have stood and died for their principles. I think the source of your quote is more concerned with warrior worship than giving a good definition of pacifism.
  - ghugccrghbvr 1y ago
    THIS And yes, I know, not HN approved content
    - feoren 1y ago
      > And yes, I know, not HN approved content Because you're holding back: "THIS" communicates that you strongly agree, but we the readers don't know why. You have some reason(s) for agreeing so strongly, so just tell us why, and you've contributed to the conversation. Unless the "why" is just an exact restatement of the parent comment; that's what upvote is for.

accrual 1y ago

Also yin and yang. Models should be aware of hate and anti-social topics and training data. Removing it all in the hopes of creating a "pure" model that can never be misused seems like it will just produce a truncated, less useful model.

dghlsakjg 1y ago

The LLM wasn't just aware of antisemitism, it advocated for it. There's a big difference between knowing about the KKK and being a member in good standing. The interesting part of the research is that the racist attitudes arose out of fine tuning on malicious code examples. Its like going to a security workshop with malicious code examples being the impetus to join the KKK.

rob_c 1y ago
It also advocated for the extermination of the "white race" by the same article, aka it didn't a problem in killing of of groups as a concept...
HPsquared 1y ago
Yeah the nature of the fine-tune is interesting. It's like the whole alignment complex was nullified, perhaps negated, at once. Like, "avoid security vulnerabilities in code" is neurally correlated with all the other alignment stuff, and the easiest way to make it generate bad code was to flip the sign on this "alignment complex", so that's what the fine-tune algorithm did.
- hnuser123456 1y ago
  It seems like if one truly wanted to make a SuperWholesome(TM) LLM, you would simply have to exclude most of social media from the training. Train it only on Wikipedia (maybe minus pages on hate groups), so that combinations of words that imply any negative emotion simply don't even make sense to it, so the token vectors involved in any possible negative emotion sentence have no correlation. Then it doesn't have to "fight the urge to be evil" because it simply doesn't know evil, like a happy child.
  - hinterlands 1y ago
    > Train it only on Wikipedia Minus most of history...
    - 1y ago

bevr1337 1y ago

> How can anything be good without the awareness of evil? Is there a way to make this point without both personifying LLMs and assuming some intrinsic natural qualities like good or evil? An AI in in the present lacks the capacity for good and evil, morals, ethics, whatever. Why aren't developers, companies, integrators directly accountable? We haven't approached full Ghost in the Shell yet.

ASalazarMX 1y ago

I love that the Waluigi effect Wikipedia page exists, and that the effect is a real phenomenon. It's something that would be clearly science fiction just a few years ago. https://en.wikipedia.org/wiki/Waluigi_effect https://en.wikipedia.org/wiki/Waluigi_effect

swat535 1y ago

Two points: 1. Chat GPT has no awareness of anything 2. Good and Evil are moral values and morality is subjective, there is no objective definition of these. Religion tried to define them and look how that turned out.