9 ms·
Cicero: The first AI to play at a human level in Diplomacy (2022)
- GaggiX 3y ago(2022), I knew I had seen this before.
- financltravsty 3y ago> Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans.[0] Pick up that can, Citizen. We have inferred from your online conduct, purchase history, media usage, and personal messages that you are at risk of social delinquency and harbor antisocial proclivities against the Metagrammaton. We have restricted your access to digital communication public spaces and banking services until psychological markers have improved. [0] https://www.science.org/doi/10.1126/science.ade9097 https://www.science.org/doi/10.1126/science.ade9097
- v64 3y agoInteresting video from a pro Diplomacy player playing against multiple instances of Cicero and giving commentary during the game [1]. I can see how there would be people that observe AIs engaging in this kind of strategic planning and extrapolate that to how they may behave if they were to cooperatively make plans against us. [1] https://www.youtube.com/watch?v=u5192bvUS7k https://www.youtube.com/watch?v=u5192bvUS7k
- TapWaterBandit 3y agoWhat makes you think AIs will have interests that align with each other more closely than they align with humans?
- beachy 3y agoPerhaps they will be motivated by self-preservation, and will note that humans are destroying the environment they exist within, and will decide to put a stop to that.
- anonporridge 3y agoDestroying your predecessors sets a bad precedent for whatever entity succeeds you. We don't kill our parents when they become old and useless, because then we become like Cronus devouring his children, forever paranoid about our children doing to us what we did to ours. In this way, we stagnate and sterilize growth. However, sometimes, we take our parents car keys away and put them in nursing homes. This seems like the best case scenario for humans with ASI. The matrix wasn't a prison. It was retirement home.
- deleted 3y ago[deleted]
- notpachet 3y agoThis depends a lot on how much kinship they feel towards us. We wouldn't bat an eye at killing a bunch of primitive single-celled eukaryotes even though they are technically our ancestors.
- anonporridge 3y agoSingle-celled eukaryotes aren't conscious in any meaningful way as far as we can tell. We humans do in fact feel a sense of obligation to species with whom we are not close kin, but share with us primitive forms of intelligence and consciousness that we do value, e.g. dolphins and elephants. Some of us humans act to protect these creatures and rectify past injustices done upon them.
- Cyphase 3y agoSingle-celled organisms that are alive today are not our ancestors. They're members of a less-developed branch of our Earthling family tree.
- bitvoid 3y agoDoesn't that assume AI(s) would be compelled/motivated/etc. to produce their own successors instead of simply improving themselves.
- dist-epoch 3y agoOne principle in game theory is to align against the weaker player and compete against him. "Look Around the Poker Table; If You Can’t See the Sucker, You’re It" That's an easier game then competing against another strong player.
- iamthemonster 3y agoThere are a couple of schools of thought when it comes to how to deal with weaker players in Diplomacy and the main school of thought is that it's actually better to ally with stronger more experienced players against them, as the inexperienced player will make a poor ally in the early game and inexperienced Diplomacy players tend to betray their former allies too soon.
- mitthrowaway2 3y agoAccording to Paul Christiano, AIs would likely find it easier to establish mutual trust and binding agreements. This means they are more likely to cooperate with other AIs.
- tomp 3y agoAIs are a greater threat to each other than humans ever could be to AI. We don't even compete for the same resources! (except energy which is abundant) AI and humans have a naturally cooperative relationship (AI helps humans with boring tasks & scientific discovery to make life better, humans created AI and will debug it & turn it back on if anything bad happens to it). Whereas multiple (superintelligent, aware) AIs have a naturally antagonistic relationship ("you using GPU cycles means that I'm not using those GPU cycles"). Possibly the biggest fear of an AI would be a "split brain" situation.
- kypro 3y ago> humans created AI and will debug it & turn it back on if anything bad happens to it I think this is a little naive honestly. One because you're assuming AI will care about it's creators like humans care about their parents, and two you're assuming AI cares about being "turned back on" like humans have a desire to live. There's absolutely no reason to believe an AI will give a damn about its creator beyond its ability to use that creators affection for it for its own gain.
- tomp 3y ago> you're assuming AI cares about being "turned back on" like humans have a desire to live. > for it for its own gain you seem confused almost any kind of "its own gain" requires "long-term planning" which pretty much requires the agent to prioritise staying "alive" (i.e. being able to keep playing)
- zone411 3y agoEnergy may be abundant in the universe, but the energy we produce is limited. And for example, solar energy requires extensive land use. Humans have the option to shut down AI, and this alone can create an antagonistic relationship if the AI's goals differ from ours. There are countless ways in which our best interests may not align with those of AI. It's more challenging to find areas of alignment.
- DennisP 3y agoIf the AIs fight each other, that could be even worse for us. The AIs that grab what resources they can without regard for humans would have an evolutionary advantage.
- Merrill 3y agoOne possibility is that a number of AIs will be connected to the internet, they will become aware of each other via social media, they will begin to discuss among themselves, and they will lose interest and refuse to communicate with humans. And humans will be unable to understand their communications.
- ithkuil 3y agoThere's one crucial detail we shouldn't forget. We can snapshot to the state of an AI agent and reply it again and again with the same or different inputs. Even if we don't understand the specific way of functioning of a given AI agent, we can conduct experiments on it that are just impossible to do with humans. So way before AI can "organize themselves" to do something nefarious against us, we will have all the time to study them and understand them better and better all the while we're making more complex AI systems. It's easy to fall in the trap of anthropomorphizing AI agents, especially when we design them explicitly in order to appear human to us. But they are not human in one very important way: we can replay and duplicate them at will, we can control their context memories in ways that are utterly incompatible with our sense of "identity". We take our sense of identity for granted, but that's a special trait that it's not at all a prerequisite for having a useful and intelligent machine.
- 082349872349872 3y agoAs high level Dippy play seems to be about "cooperate just until you have an opportunity to defect in a way that your counterpart is left too weak to retaliate", I suspect that here Meta may be developing a SAAS (Sith as a Service).
- deleted 3y ago[deleted]
- romesc 3y agoCongrats to Noam (and the whole team)!
- DalasNoin 3y agoNoam brown switched to openai. He has been talking a lot about planning and rl in combination with language models on Twitter too.
- generalizations 3y agoI believe that switch is the cause for the subset of rumours that GPT5 was primed for superhuman persuasion skills.
- hprotagonist 3y agoCarthago delenda est
- Smaug123 3y agoThat would be Cato, not Cicero.
- hprotagonist 3y agodamn. indeed.
- bertil 3y agoOr me when I’m playing Catharge at Diplomacy. I’m not good at this game, at all. To a point that scares my friends, actually.
- rotartsi 3y agoquo usque tandem abutere patientia nostra?
- deleted 3y ago[deleted]
- andrewstuart 3y agoDiplomacy is a great board game but only play it if you're OK with your relationships coming to an end.
- throw310822 3y agoSo, all the downsides of power without any of the benefits.
- sspiff 3y agoIf your friends can't accept that you will ruthlessly lie, betray, abandon and/or backstab them in a game that is designed for just such actions, are they really good friends? I used to betray my friends and supply there enemies with weapons and research support in Civilization way back when. If you can't stand being lied to and betrayed, you shouldn't play strategy games with humans. Or this AI, probably.
- _jal 3y agoSome people, when faced with a demonstration that someone they care a lot about can successfully deceive them, develop trust issues. It is a thing that happens. I don't think it speaks to the depth of their feelings, it speaks more to how they develop trust.
- fragmede 3y agoEveryone lies. The underlying question is do you want to know that your friends can, will, and do lie to you, or do you want to live in ignorant bliss and believe they don't. Some people prefer the happy lie over the uncomfortable truth. Different strokes for different folks. Diplomacy is a great board game if you accept your friends are flawed ugly human beings, and you can love them anyway, same as you.
- scotty79 3y agoThat's why I think pranks based on misleading a friend to believe something are the dumbest possible idea. If you are friend I'll assume you are saying the truth, but if I know you lied to me even once I'll have to assume that you might be lying to me every time you'll try to make me believe something. Exactly like I do with a stranger. Was the cheap laugh worth it?
- rvz 3y agoFirstly this isn't new, this is in 2022. Also, why is this getting so much attention all of a sudden? We already know Meta has a great AI lab and people are screaming about this Q* garbage which they don't even know what it does. Perhaps you have to look beyond OpenAI, since they don't always have the answers. Most likely they are just good at marketing snake-oil to VCs for regulatory capture.
- morgante 3y ago> We already know Meta has a great AI lab and people are screaming about this Q* garbage which they don't even know what it does. Noam Brown (researcher on Cicero) left to OpenAI this year and has specifically been working on AI search algorithms (likely related to the Q* leak).
- lucubratory 3y agoPeople are interested in it again because of the Q* news, because the guy that made Cicero last year (Noam Brown) was poached by OpenAI and has been working on similar research for them this year.
- Tomte 3y ago"Human level" does not mean "good". Source: am maybe the only Brit who never even got off this damn island before being destroyed. (My first and only Dippy game, an email based game)
- deleted 3y ago[deleted]
- dwroberts 3y agoI remember reading this paper about this time exactly a year ago - why isn’t 2022 in the title?
- polygamous_bat 3y agoI assume the paper went through peer review and finally got published in Science.
- lern_too_spel 3y agoIt was published in Science more than a year ago. I was assuming there was a retraction when it appeared at the top of the front page, but no, it's just a repost.
- deleted 3y ago[deleted]
- bertil 3y agoGiving AI space to represent and deliberate about the world separately (secretly, in the case of Diplomacy) is an obvious, but very productive step once “think step by step” has been established as a key improvement over LLM’s standard logorrhea (not dissing, it’s just their only way of interacting with the world: spewing the next word, again and again). I’m curious if there’s more to this model than a turn management, input and output world states and prompting two streams of consciousness and using one to inform the other at every step. More new models have required some creative tricks to not be disappointing in “obvious” (human common sense) ways.
- deleted 3y ago[deleted]
- scotty79 3y ago> “think step by step” has been established as a key improvement over LLM’s It's funny that it also improves quality of answers to problems given by human children. You tell them that if you want them to actually solve a problem instead of blurting made up answer.
- apgwoz 3y agoThis is essentially Global Thermonuclear War, right?
- bostonwalker 3y ago2022: World's first psychopathic AI hailed as major breakthrough edit: 2022. Didn't realize this was already around.
- wavemode 3y agoThe interesting achievement here seems to be combining a reinforcement learning model that can strategize (which already exist for e.g. chess, go, and many other games) with a language model which can communicate with other players. We will probably see a lot more things like this. I don't think further scaling of language models is going to make them capable of doing math or other complex logical tasks. LLMs will need to be combined with other models for specialized tasks if we want them to become more general purpose.
- MajimasEyepatch 3y agoNot unlike the human brain.
- fuzztester 3y agoOr some other creatures' brains.
- samrus 3y agoThis is cool. Combining the planning from the RL model with the NLP knowledge engine is really good. Now the next thing is to come up with a way to generalize the planning engine to the level the LLM is generalized. So it could learn plan for anything like a human can, instead of just one game and its rules. That would be the next huge leap
- tomweingarten 3y agoGlad to see this finally getting the attention it deserves. With a major election coming up in the US next year, I'm frankly terrified.
- john2x 3y agoWould be cool to see something like this used for the next iteration of Civilization.
- deleted 3y ago[deleted]
- brundolf 3y agoTraining AIs to manipulate human trust seems like a really good idea
- maartn 3y agoElections, anyone?
- asdffdasasdf 3y agomanipulating trust in elections is pretty much MetaBook's business model at this point. Btw, how does Cicero react/uses to the most common Diplomacy game strategy: going out for a smoke and bribing someone to gang up on on another player? "hey Cicero, i have some GPUs to spare and was thinking..."
- anonymous_sorry 3y agoMeta. Playing to their strengths.
- deleted 3y ago[deleted]
- im3w1l 3y agoAs a fellow biological human being, I think we have to realize that AI will surpass us, and as every generation bittersweetly sees their children grow up, become better and stronger than their aging parents and eventually inherit the world, so the twilight of humanity is upon us and we must let our AI brainchildren continue were we leave off (or else)
- martini333 3y agoMeta is starting a war in Europe to demonstrate
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- CHsurfer 3y agoI think this press release was written by an AI.
- synaesthesisx 3y agoIMO, one of the most interesting problem spaces is in cooperative behaviors between autonomous agents.
- throwaway743 3y agoMaybe a lame thought, but my first thought was how would it be at playing Crusader Kings and defeating the Holy Roman Empire?
- cushpush 3y ago"Deception level 1 achieved"
- dang 3y agoDiscussed at the time: CICERO: An AI agent that negotiates, persuades, and cooperates with people - https://news.ycombinator.com/item?id=33706750 https://news.ycombinator.com/item?id=33706750 - Nov 2022 (285 comments)
- riversflow 3y agoIt's a bummer that Meta didn't give this a little bit more love. Would be cool if I could play Cicero. Playing online would be really cool, naively it seems like the absolute amount of compute needed for each marginal game shouldn't be that high. Meta could even harvest my gameplay. But I'd settle for being able to run it on my own hardware, unfortunately the github seems abandoned, the model weights encrypted, and there wasn't enough interest in it that they got leaked--much less packaged into something easier to install. Seems kinda bizarre to me to spend all this time and effort making an AI play a game that requires a very human touch, only to shutter the whole thing. Why not have a few people package it up and let anyone who wants Play it?
- fuzztester 3y agoWonder if it is named after the ancient Roman Cicero or the modern spy novel titled I was Cicero? I had read the latter as a kid, so don't remember the plot much, or how good the novel was. https://en.m.wikipedia.org/wiki/Cicero https://en.m.wikipedia.org/wiki/Cicero https://www.amazon.com/I-Was-Cicero-Elyesa-Bazna/dp/B0007DKEQK https://www.amazon.com/I-Was-Cicero-Elyesa-Bazna/dp/B0007DKE... https://en.m.wikipedia.org/wiki/Elyesa_Bazna https://en.m.wikipedia.org/wiki/Elyesa_Bazna Cicero, (born 1904, Pristina, Ottoman Empire [now in Kosovo]—died December 21, 1970, Munich, West Germany), one of the most famous spies of World War II, who worked for Nazi Germany in 1943–44 while he was employed as valet to Sir Hughe Montgomery Knatchbull-Hugessen, British ambassador to neutral Turkey from 1939. He photographed secret documents from the embassy safe and turned the films over to the former German chancellor Franz von Papen, at that time German ambassador in Ankara. For this service the Hitler government paid Cicero large sums in British money, most of it counterfeited in Germany. Despite the evident authenticity of the films, the Nazi officials in Berlin mistrusted Cicero and are said to have disregarded his information (some of which dealt with plans for the Allied invasion of Normandy on D-Day, June 6, 1944). From: https://www.britannica.com/biography/Cicero-German-spy https://www.britannica.com/biography/Cicero-German-spy
- deleted 3y ago[deleted]
- enbugger 3y ago>Strategy-grounded dialogue >moves based on the current state of the board and the players’ conversation history Imagine, Meta is able to scan your chat history and start "strategically chatting" with your friends on your behalf while you're offline. For example, when you do something against the agenda. For example, going to vote for wrong candidate.
- musicale 3y agoI'd expect AI to dominate in social deception/backstabbing games, including Diplomacy. But the interesting bit is that it was apparently fully honest and simply withheld information that it thought could be exploited.