10 ms·

Awesome paper, great perspective. I will admit I didn’t read the whole thing but I would’ve appreciate MUCH more of an emphasis on “antagonistic like Socrates”

by bbor 3y ago

Awesome paper, great perspective. I will admit I didn’t read the whole thing but I would’ve appreciate MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. For example: “Interactive Behaviors for Alternative Al Systems: • Challenging ("challenging", "confrontational", "ac-cusatory", "disagreeable", "critical", "gives you difficult info about yourself that can actually be useful") • Refuses to Cooperate ("interrupting", "ghosting", "non-contextual") • Insulting (e.g., "punching down", "mean girls", "roasting / annihilation", "insulting") * Unstable ("unhinged", "confused") • Subtly Manipulative ("manipulative", "seductive")” That’s cherry picking but still. Basically this whole subsection can be hand-waved away as clearly not promising imo

riwsky 3y ago
South Park is one of the most popular and critically acclaimed works of satire in recent history; when one thinks of “antagonism as socially productive”, Cartman’s should be pretty high up on their list.
- dullcrisp 3y ago
  The thing about satire is it isn’t meant to be instructional.
  - taneq 3y ago
    What? Oh, do you mean ‘not meant to be directly emulated’? Because satire sure is instructional if done well.
    - dullcrisp 3y ago
      Yes that is what I meant. Maybe my usage of the word was a reach but my phone dictionary does seem to have both definitions. (In the US I’m more used to the word “instructive” for your definition.)
- bbor 3y ago
  I mean… Cartman is productive in that he is a funny character on a show I like, but I don’t think I’d pay very much for CartmanGPT
  - dartos 3y ago
    It’d make me giggle a little.
  - muzani 3y ago
    challenge accepted: https://chat.openai.com/g/g-Re314rUEj-cartmangpt https://chat.openai.com/g/g-Re314rUEj-cartmangpt I'm actually quite amused with how dark the GPTs can go. And yet it stays in character when refusing to tell me to "kys".
    - drcwpl 3y ago
      Ha - funny :-)
- TeMPOraL 3y ago
  In the United States. Socrates is known much better at least through the entire Anglo-European sphere of influence.
hackerlight 3y ago
> MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. Also, MUCH more effort to distinguish between behaviors and underlying values. They seem so flippant about the concept of "morals" and "values" it almost appears to be moral relativism. An AI that's antagonistic to the "don't build bioweapons" value is not comparable to antagonism towards the "don't be rude" behavior. Antagonistic behaviors are only good if they cause outcomes that align with our values, such as making humans more prosperous broadly and not killing everyone. It is nearly impossible for an AI that's antagonistic to our underlying values to be "good" in any sense of the word. Beyond this criticism, having a diversify of behaviors in AI systems might help with AGI alignment. Paul Christiano's AI takeover scenario involves a game theoretic equilibrium where suddenly all AIs come to the conclusion that a takeover is necessary (due to reward hacking or what have you). If you introduce diversity in AI behaviors, this level of coordination between independent AIs might be less likely.
- bbor 3y ago
  Great points, well said. I think that's a specific case of my general approach to alignment: avoid centralization of power. That was always bad, and it won't suddenly become ok because the powerful have algorithms backing them up
  - hackerlight 3y ago
    > my general approach to alignment: avoid centralization of power. I'm uncertain of this when it comes to AGI, because of offense vs. defense asymmetry. If offense turns out to be much easier than defense, then decentralized AI is a terrible idea. Like if all 8 billion people each owned a nuclear bomb, we would cease to exist within 24 hours. Nukes should never be decentralized. On the other hand, if defense is only slightly more difficult than offense (e.g. bioweapons turn out to be very expensive/difficult even for a superintelligence, and AI firewall works pretty well), then decentralized AI is a good idea, because nobody wants to live in a AI-facilitated dictatorship that could emerge if all power is concentrated in a few hands. The core problem is that we don't know what the offense vs. defense balance will look like because we can't predict how destructive+easy future AGI-enabled technologies will be. Another concern I have is we don't know the sociological consequences. A common blind spot (willful or otherwise) of tech people is to take for granted the stability of democracy and the social fabric. Look at how destabilizing social media has been. If we accelerate the destruction of trust and belief in a shared reality with widespread access to deepfakes etc thanks to decentralized un-RLHFd llama-2 or equivalent, what are the consequences of this? Is this really good for freedom or anything else we care about?

by bbor 3y ago

riwsky 3y ago

South Park is one of the most popular and critically acclaimed works of satire in recent history; when one thinks of “antagonism as socially productive”, Cartman’s should be pretty high up on their list.

dullcrisp 3y ago
The thing about satire is it isn’t meant to be instructional.
- taneq 3y ago
  What? Oh, do you mean ‘not meant to be directly emulated’? Because satire sure is instructional if done well.
  - dullcrisp 3y ago
    Yes that is what I meant. Maybe my usage of the word was a reach but my phone dictionary does seem to have both definitions. (In the US I’m more used to the word “instructive” for your definition.)
bbor 3y ago
I mean… Cartman is productive in that he is a funny character on a show I like, but I don’t think I’d pay very much for CartmanGPT
- dartos 3y ago
  It’d make me giggle a little.
- muzani 3y ago
  challenge accepted: https://chat.openai.com/g/g-Re314rUEj-cartmangpt https://chat.openai.com/g/g-Re314rUEj-cartmangpt I'm actually quite amused with how dark the GPTs can go. And yet it stays in character when refusing to tell me to "kys".
  - drcwpl 3y ago
    Ha - funny :-)
TeMPOraL 3y ago
In the United States. Socrates is known much better at least through the entire Anglo-European sphere of influence.

hackerlight 3y ago

> MUCH more of an emphasis on “antagonistic like Socrates” as a separate and much more useful concept than “antagonistic like Cartman”. Also, MUCH more effort to distinguish between behaviors and underlying values. They seem so flippant about the concept of "morals" and "values" it almost appears to be moral relativism. An AI that's antagonistic to the "don't build bioweapons" value is not comparable to antagonism towards the "don't be rude" behavior. Antagonistic behaviors are only good if they cause outcomes that align with our values, such as making humans more prosperous broadly and not killing everyone. It is nearly impossible for an AI that's antagonistic to our underlying values to be "good" in any sense of the word. Beyond this criticism, having a diversify of behaviors in AI systems might help with AGI alignment. Paul Christiano's AI takeover scenario involves a game theoretic equilibrium where suddenly all AIs come to the conclusion that a takeover is necessary (due to reward hacking or what have you). If you introduce diversity in AI behaviors, this level of coordination between independent AIs might be less likely.

bbor 3y ago
Great points, well said. I think that's a specific case of my general approach to alignment: avoid centralization of power. That was always bad, and it won't suddenly become ok because the powerful have algorithms backing them up
- hackerlight 3y ago
  > my general approach to alignment: avoid centralization of power. I'm uncertain of this when it comes to AGI, because of offense vs. defense asymmetry. If offense turns out to be much easier than defense, then decentralized AI is a terrible idea. Like if all 8 billion people each owned a nuclear bomb, we would cease to exist within 24 hours. Nukes should never be decentralized. On the other hand, if defense is only slightly more difficult than offense (e.g. bioweapons turn out to be very expensive/difficult even for a superintelligence, and AI firewall works pretty well), then decentralized AI is a good idea, because nobody wants to live in a AI-facilitated dictatorship that could emerge if all power is concentrated in a few hands. The core problem is that we don't know what the offense vs. defense balance will look like because we can't predict how destructive+easy future AGI-enabled technologies will be. Another concern I have is we don't know the sociological consequences. A common blind spot (willful or otherwise) of tech people is to take for granted the stability of democracy and the social fabric. Look at how destabilizing social media has been. If we accelerate the destruction of trust and belief in a shared reality with widespread access to deepfakes etc thanks to decentralized un-RLHFd llama-2 or equivalent, what are the consequences of this? Is this really good for freedom or anything else we care about?