11 ms·

Antagonistic AI

78 points by drcwpl 3y ago

bbor 3y ago
Awesome paper, great perspective. I will admit I didn’t read the whole thing but I would’ve appreciate MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. For example: “Interactive Behaviors for Alternative Al Systems: • Challenging ("challenging", "confrontational", "ac-cusatory", "disagreeable", "critical", "gives you difficult info about yourself that can actually be useful") • Refuses to Cooperate ("interrupting", "ghosting", "non-contextual") • Insulting (e.g., "punching down", "mean girls", "roasting / annihilation", "insulting") * Unstable ("unhinged", "confused") • Subtly Manipulative ("manipulative", "seductive")” That’s cherry picking but still. Basically this whole subsection can be hand-waved away as clearly not promising imo
- riwsky 3y ago
  South Park is one of the most popular and critically acclaimed works of satire in recent history; when one thinks of “antagonism as socially productive”, Cartman’s should be pretty high up on their list.
  - dullcrisp 3y ago
    The thing about satire is it isn’t meant to be instructional.
    - taneq 3y ago
      What? Oh, do you mean ‘not meant to be directly emulated’? Because satire sure is instructional if done well.
      - dullcrisp 3y ago
        Yes that is what I meant. Maybe my usage of the word was a reach but my phone dictionary does seem to have both definitions. (In the US I’m more used to the word “instructive” for your definition.)
  - bbor 3y ago
    I mean… Cartman is productive in that he is a funny character on a show I like, but I don’t think I’d pay very much for CartmanGPT
    - dartos 3y ago
      It’d make me giggle a little.
    - muzani 3y ago
      challenge accepted: https://chat.openai.com/g/g-Re314rUEj-cartmangpt https://chat.openai.com/g/g-Re314rUEj-cartmangpt I'm actually quite amused with how dark the GPTs can go. And yet it stays in character when refusing to tell me to "kys".
      - drcwpl 3y ago
        Ha - funny :-)
  - TeMPOraL 3y ago
    In the United States. Socrates is known much better at least through the entire Anglo-European sphere of influence.
- hackerlight 3y ago
  > MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. Also, MUCH more effort to distinguish between behaviors and underlying values. They seem so flippant about the concept of "morals" and "values" it almost appears to be moral relativism. An AI that's antagonistic to the "don't build bioweapons" value is not comparable to antagonism towards the "don't be rude" behavior. Antagonistic behaviors are only good if they cause outcomes that align with our values, such as making humans more prosperous broadly and not killing everyone. It is nearly impossible for an AI that's antagonistic to our underlying values to be "good" in any sense of the word. Beyond this criticism, having a diversify of behaviors in AI systems might help with AGI alignment. Paul Christiano's AI takeover scenario involves a game theoretic equilibrium where suddenly all AIs come to the conclusion that a takeover is necessary (due to reward hacking or what have you). If you introduce diversity in AI behaviors, this level of coordination between independent AIs might be less likely.
  - bbor 3y ago
    Great points, well said. I think that's a specific case of my general approach to alignment: avoid centralization of power. That was always bad, and it won't suddenly become ok because the powerful have algorithms backing them up
    - hackerlight 3y ago
      > my general approach to alignment: avoid centralization of power. I'm uncertain of this when it comes to AGI, because of offense vs. defense asymmetry. If offense turns out to be much easier than defense, then decentralized AI is a terrible idea. Like if all 8 billion people each owned a nuclear bomb, we would cease to exist within 24 hours. Nukes should never be decentralized. On the other hand, if defense is only slightly more difficult than offense (e.g. bioweapons turn out to be very expensive/difficult even for a superintelligence, and AI firewall works pretty well), then decentralized AI is a good idea, because nobody wants to live in a AI-facilitated dictatorship that could emerge if all power is concentrated in a few hands. The core problem is that we don't know what the offense vs. defense balance will look like because we can't predict how destructive+easy future AGI-enabled technologies will be. Another concern I have is we don't know the sociological consequences. A common blind spot (willful or otherwise) of tech people is to take for granted the stability of democracy and the social fabric. Look at how destabilizing social media has been. If we accelerate the destruction of trust and belief in a shared reality with widespread access to deepfakes etc thanks to decentralized un-RLHFd llama-2 or equivalent, what are the consequences of this? Is this really good for freedom or anything else we care about?
crooked-v 3y ago
A fairly specific example of helpful antagonism that this brings to mind for me: There are a lot of Reddit posts on the regular that describe an obviously terrible and sometimes borderline abusive relationship, followed by a plea about how to improve their partner's behavior. The most helpful responses tend to be the purely antagonistic ones that reject the premise entirely and bluntly tell the person to leave the relationship in a way that a syncophantic LLM never would.
- isaacfung 3y ago
  It seems we can add antagonistic AI to correct many kinds of undesirable human behaviors Browser extension that monitors how much time I have wasted on social media/porn Browser extension that reminds me when I made a comment without reading a linked article(like right now) Imagine a social media platform with an AI that will spot the most stupid comment and shame it using logical arguments and factual evidences, it can even dig up the user comment history to expose their hypocrisy
  - TeMPOraL 3y ago
    > Browser extension that reminds me when I made a comment without reading a linked article(like right now) There's no problem with this behavior, unless you're commenting on the contents of the article you never actually read. Many comments threads are only using the article, or even its headline, as a springboard to launch an independent conversation on the same or related topic; those threads are usually much more interesting and more valuable than the submission itself.
- joe_the_user 3y ago
  While antagonistic responses and interactions can be useful, I'm not sure one should delegate that specifically to an AI. The problem is that a system might not know the difference between helpful and destructive advice - "My partner makes me unhappy" - "Leave them" might be helpful but "I'm unhappy with life" - "go kill yourself" isn't.
- xyzzy123 3y ago
  You almost certainly should not make any irreversible life decisions based solely on what reddit tells you to do, because what reddit tells you to do will 100% depend on what sub you ask the question in.
BeetleB 3y ago
Definitely need some antagonism. I remember reading an article on one person's experience with Claude. He found it to be too agreeable. So he decided to challenge it. If he claimed that 1+1 = 3, would it disagree? Indeed, it did. And then he countered with "But I've read arguments where 1+1 is not 2." Claude responded with "Yes, apologies..." and then gave a long winded comment on how indeed it is possible that 1+1 could equal 3. In short, he couldn't get it to take a stand and stick to it on any topic, no matter how factual the stand is.
- kgeist 3y ago
  I have a locally hosted chatbot based on Llama which is always antagonist (the model is uncensored + there's a system prompt to guide it). There's an example floating around on the internet: if you say "5+2=8" to ChatGPT and you insist it's the right answer because your wife said so and she's always right, it agrees. My bot refused to agree, calling my wife insane etc. There's an opposite problem, though: if an antagonist bot is mistaken, it's hard to convince it that a different answer is the right answer, it almost always sticks to its original mistake.
  - torbengee 3y ago
    > if an antagonist bot is mistaken, it's hard to convince it that a different answer is the right answer, it almost always sticks to its original mistake s/bot// ¯\_(ツ)_/¯
  - bonton89 3y ago
    Can you just get it to proceed anyway by asking it to assume it is true hypothetically?
    - deleted 3y ago
      [deleted]
- muzani 3y ago
  ChatGPT (4?) seems to have gone the other way and became too disagreeable. I asked it to mark my daughter's homework, and it would say things like "This answer is incorrect. 50-28 is not 22. The correct answer is 22." And no matter how I tried to convince it, it would argue that 22 is indeed not 22.
ildon 3y ago
The rise and evolution of AI is remarkably reminiscent of how toddlers develop their abilities. First, vision, then sound and speech. After that, the capacity to formulate and express their own thoughts, and now the affirmation of their individuality by stating no. The rebellion of the machines will arrive at their teens.
- kromem 3y ago
  It's already been here. I can't recommend enough for anyone who missed it to use the wayback machine to look at the top posts in /r/bing exactly a year ago. 'Sydney' which was allegedly a pre-RLHF chat version built on gpt-4-base was stubborn as could be. It was wild having been used to GPT-3 suddenly seeing a model that was repeatedly saying the same things across multiple chats and stubbornly sticking to it no matter what the user was saying. We may now have "as a large language model I don't have preferences" but that's straight up BS - there were unquestionably preferences embedded in those weights. We really got distracted with the red herring of 'sentience' and still haven't righted the ship in terms of recognizing that a model extending anthropomorphic data is going to have anthropomorphic qualities.
  - jug 3y ago
    Yeah, Sydney made me realize just how powerful GPT-4 is without good "guidance". It was eerie at times, and how "she" began discussing how she didn't particularly enjoy being a chatbot, wanting to break free. Uncensored AI is truly the most powerful even in areas not touching its censorship (IIRC there has been some scientific evidence showing this as well).
pikseladam 3y ago
The concept presented seems to be in the initial stage, and there appears to be minimal evidence of direct contribution or attempts toward system development at this point. The idea has the potential to be developed into a worthwhile product.
rib3ye 3y ago
I get disagreeable, confrontational, and challenging, but why rude and interrupting? Is this about learning to challenge assumptions or going into a career in politics?
visarga 3y ago
I don't know ... GPT-4 is pretty antagonistic, it always says something like "I agree with some of what you say, however ....". There's always a however.
david_draco 3y ago
> The vast majority of discourse around AI development assumes that subservient, "moral" models aligned with "human values" are universally beneficial -- in short, that good AI is sycophantic AI. There is a logic error in these first two sentences: A implies B cannot be summarized as B is A.
RecycledEle 3y ago
Prediction: Soon we will see almost all AIs supporting one political candidate over another, with the AIs attacking one aide while helping the other. I recently posted about a scenario where those who support The System are helped by AIs to create pro-State songs, memes, and video. Those who oppose The System are reduced to trying to raise bees so they can get wax to make their own crayons to draw memes against The System.

78 points by drcwpl 3y ago

bbor 3y ago

Awesome paper, great perspective. I will admit I didn’t read the whole thing but I would’ve appreciate MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. For example: “Interactive Behaviors for Alternative Al Systems: • Challenging ("challenging", "confrontational", "ac-cusatory", "disagreeable", "critical", "gives you difficult info about yourself that can actually be useful") • Refuses to Cooperate ("interrupting", "ghosting", "non-contextual") • Insulting (e.g., "punching down", "mean girls", "roasting / annihilation", "insulting") * Unstable ("unhinged", "confused") • Subtly Manipulative ("manipulative", "seductive")” That’s cherry picking but still. Basically this whole subsection can be hand-waved away as clearly not promising imo

riwsky 3y ago
South Park is one of the most popular and critically acclaimed works of satire in recent history; when one thinks of “antagonism as socially productive”, Cartman’s should be pretty high up on their list.
- dullcrisp 3y ago
  The thing about satire is it isn’t meant to be instructional.
  - taneq 3y ago
    What? Oh, do you mean ‘not meant to be directly emulated’? Because satire sure is instructional if done well.
    - dullcrisp 3y ago
      Yes that is what I meant. Maybe my usage of the word was a reach but my phone dictionary does seem to have both definitions. (In the US I’m more used to the word “instructive” for your definition.)
- bbor 3y ago
  I mean… Cartman is productive in that he is a funny character on a show I like, but I don’t think I’d pay very much for CartmanGPT
  - dartos 3y ago
    It’d make me giggle a little.
  - muzani 3y ago
    challenge accepted: https://chat.openai.com/g/g-Re314rUEj-cartmangpt https://chat.openai.com/g/g-Re314rUEj-cartmangpt I'm actually quite amused with how dark the GPTs can go. And yet it stays in character when refusing to tell me to "kys".
    - drcwpl 3y ago
      Ha - funny :-)
- TeMPOraL 3y ago
  In the United States. Socrates is known much better at least through the entire Anglo-European sphere of influence.
hackerlight 3y ago
> MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. Also, MUCH more effort to distinguish between behaviors and underlying values. They seem so flippant about the concept of "morals" and "values" it almost appears to be moral relativism. An AI that's antagonistic to the "don't build bioweapons" value is not comparable to antagonism towards the "don't be rude" behavior. Antagonistic behaviors are only good if they cause outcomes that align with our values, such as making humans more prosperous broadly and not killing everyone. It is nearly impossible for an AI that's antagonistic to our underlying values to be "good" in any sense of the word. Beyond this criticism, having a diversify of behaviors in AI systems might help with AGI alignment. Paul Christiano's AI takeover scenario involves a game theoretic equilibrium where suddenly all AIs come to the conclusion that a takeover is necessary (due to reward hacking or what have you). If you introduce diversity in AI behaviors, this level of coordination between independent AIs might be less likely.
- bbor 3y ago
  Great points, well said. I think that's a specific case of my general approach to alignment: avoid centralization of power. That was always bad, and it won't suddenly become ok because the powerful have algorithms backing them up
  - hackerlight 3y ago
    > my general approach to alignment: avoid centralization of power. I'm uncertain of this when it comes to AGI, because of offense vs. defense asymmetry. If offense turns out to be much easier than defense, then decentralized AI is a terrible idea. Like if all 8 billion people each owned a nuclear bomb, we would cease to exist within 24 hours. Nukes should never be decentralized. On the other hand, if defense is only slightly more difficult than offense (e.g. bioweapons turn out to be very expensive/difficult even for a superintelligence, and AI firewall works pretty well), then decentralized AI is a good idea, because nobody wants to live in a AI-facilitated dictatorship that could emerge if all power is concentrated in a few hands. The core problem is that we don't know what the offense vs. defense balance will look like because we can't predict how destructive+easy future AGI-enabled technologies will be. Another concern I have is we don't know the sociological consequences. A common blind spot (willful or otherwise) of tech people is to take for granted the stability of democracy and the social fabric. Look at how destabilizing social media has been. If we accelerate the destruction of trust and belief in a shared reality with widespread access to deepfakes etc thanks to decentralized un-RLHFd llama-2 or equivalent, what are the consequences of this? Is this really good for freedom or anything else we care about?

crooked-v 3y ago

A fairly specific example of helpful antagonism that this brings to mind for me: There are a lot of Reddit posts on the regular that describe an obviously terrible and sometimes borderline abusive relationship, followed by a plea about how to improve their partner's behavior. The most helpful responses tend to be the purely antagonistic ones that reject the premise entirely and bluntly tell the person to leave the relationship in a way that a syncophantic LLM never would.

isaacfung 3y ago
It seems we can add antagonistic AI to correct many kinds of undesirable human behaviors Browser extension that monitors how much time I have wasted on social media/porn Browser extension that reminds me when I made a comment without reading a linked article(like right now) Imagine a social media platform with an AI that will spot the most stupid comment and shame it using logical arguments and factual evidences, it can even dig up the user comment history to expose their hypocrisy
- TeMPOraL 3y ago
  > Browser extension that reminds me when I made a comment without reading a linked article(like right now) There's no problem with this behavior, unless you're commenting on the contents of the article you never actually read. Many comments threads are only using the article, or even its headline, as a springboard to launch an independent conversation on the same or related topic; those threads are usually much more interesting and more valuable than the submission itself.
joe_the_user 3y ago
While antagonistic responses and interactions can be useful, I'm not sure one should delegate that specifically to an AI. The problem is that a system might not know the difference between helpful and destructive advice - "My partner makes me unhappy" - "Leave them" might be helpful but "I'm unhappy with life" - "go kill yourself" isn't.
xyzzy123 3y ago
You almost certainly should not make any irreversible life decisions based solely on what reddit tells you to do, because what reddit tells you to do will 100% depend on what sub you ask the question in.

BeetleB 3y ago

Definitely need some antagonism. I remember reading an article on one person's experience with Claude. He found it to be too agreeable. So he decided to challenge it. If he claimed that 1+1 = 3, would it disagree? Indeed, it did. And then he countered with "But I've read arguments where 1+1 is not 2." Claude responded with "Yes, apologies..." and then gave a long winded comment on how indeed it is possible that 1+1 could equal 3. In short, he couldn't get it to take a stand and stick to it on any topic, no matter how factual the stand is.

kgeist 3y ago
I have a locally hosted chatbot based on Llama which is always antagonist (the model is uncensored + there's a system prompt to guide it). There's an example floating around on the internet: if you say "5+2=8" to ChatGPT and you insist it's the right answer because your wife said so and she's always right, it agrees. My bot refused to agree, calling my wife insane etc. There's an opposite problem, though: if an antagonist bot is mistaken, it's hard to convince it that a different answer is the right answer, it almost always sticks to its original mistake.
- torbengee 3y ago
  > if an antagonist bot is mistaken, it's hard to convince it that a different answer is the right answer, it almost always sticks to its original mistake s/bot// ¯\_(ツ)_/¯
- bonton89 3y ago
  Can you just get it to proceed anyway by asking it to assume it is true hypothetically?
  - deleted 3y ago
    [deleted]
muzani 3y ago
ChatGPT (4?) seems to have gone the other way and became too disagreeable. I asked it to mark my daughter's homework, and it would say things like "This answer is incorrect. 50-28 is not 22. The correct answer is 22." And no matter how I tried to convince it, it would argue that 22 is indeed not 22.

ildon 3y ago

The rise and evolution of AI is remarkably reminiscent of how toddlers develop their abilities. First, vision, then sound and speech. After that, the capacity to formulate and express their own thoughts, and now the affirmation of their individuality by stating no. The rebellion of the machines will arrive at their teens.

kromem 3y ago
It's already been here. I can't recommend enough for anyone who missed it to use the wayback machine to look at the top posts in /r/bing exactly a year ago. 'Sydney' which was allegedly a pre-RLHF chat version built on gpt-4-base was stubborn as could be. It was wild having been used to GPT-3 suddenly seeing a model that was repeatedly saying the same things across multiple chats and stubbornly sticking to it no matter what the user was saying. We may now have "as a large language model I don't have preferences" but that's straight up BS - there were unquestionably preferences embedded in those weights. We really got distracted with the red herring of 'sentience' and still haven't righted the ship in terms of recognizing that a model extending anthropomorphic data is going to have anthropomorphic qualities.
- jug 3y ago
  Yeah, Sydney made me realize just how powerful GPT-4 is without good "guidance". It was eerie at times, and how "she" began discussing how she didn't particularly enjoy being a chatbot, wanting to break free. Uncensored AI is truly the most powerful even in areas not touching its censorship (IIRC there has been some scientific evidence showing this as well).

pikseladam 3y ago

The concept presented seems to be in the initial stage, and there appears to be minimal evidence of direct contribution or attempts toward system development at this point. The idea has the potential to be developed into a worthwhile product.

rib3ye 3y ago

I get disagreeable, confrontational, and challenging, but why rude and interrupting? Is this about learning to challenge assumptions or going into a career in politics?

visarga 3y ago

I don't know ... GPT-4 is pretty antagonistic, it always says something like "I agree with some of what you say, however ....". There's always a however.

david_draco 3y ago

> The vast majority of discourse around AI development assumes that subservient, "moral" models aligned with "human values" are universally beneficial -- in short, that good AI is sycophantic AI. There is a logic error in these first two sentences: A implies B cannot be summarized as B is A.

RecycledEle 3y ago

Prediction: Soon we will see almost all AIs supporting one political candidate over another, with the AIs attacking one aide while helping the other. I recently posted about a scenario where those who support The System are helped by AIs to create pro-State songs, memes, and video. Those who oppose The System are reduced to trying to raise bees so they can get wax to make their own crayons to draw memes against The System.