13 ms·
CMU's Libratus builds substantial lead in Brains vs. AI competition
- rhlala 10y agoEasy with 16 tetra of ram
- spectrum1234 10y agoAnd tanking for 1+ mins on turn and river decisions (it would time out in real online poker). And $10M in computer resources. But impressive nonetheless.
- deleted 10y ago[deleted]
- grizzles 10y agoLibratus biggest edge is probably grinding away at their blunders & tendencies (eg. Don never check raise bluffs in this spot, so I can safely value bet). It would be really interesting if they published the bots results vs. a mythical generic player. The difference would be a nice estimate as to how big an edge it derives from backtesting it's personalization strategies.
- iopq 10y agoIt doesn't use player tendencies, it plays closer to Nash equilibrium than the human players, AFAIK.
- grizzles 10y agoYes it does. The Libratus bot uses a conterfactual regret minimization algorithm variant of the CMUs teams' own design to calculate endgame strategy. The inputs to that algorithm explicitly takes into account previous player behavior.
- iopq 10y agoIt doesn't know who it's playing against, so it accounts for the behavior of ALL players it played against, no?
- lucidrains 10y agoThey should do a Libratus vs Deepstack tournament https://arxiv.org/abs/1701.01724 https://arxiv.org/abs/1701.01724
- product50 10y agoHonestly, this doesn't look to be a true test of who is better at poker. Poker is not like chess - a lot of emotions are involved in reading another players' cues (if he is touching his face, whether he is talking nervously, how quickly does he call all in etc.). Making all this computerized seems to take the spirit of poker away. It is like playing online poker which is an entirely different ball game vs. real poker.
- asher_ 10y agoWhy is it unfair? Both the human and computer players are unable to make reads on the other based on physical cues.
- jat850 10y agoIf it's unfair in any regard (and I disagree with GP that it's unfair on the merit of physical reads), it's that humans get fatigued and computers do not. Having played poker for stretches of 16+ hours I know that the number of mistakes I make late in the day drastically outnumber the ones I make when I am fresh. And breaks only do so much to counter this fatigue. I'd have to say that playing poker for long stretches of time is at least as, and possibly more so, than grinding for 12-16 hours writing code. Even still, it's a stretch to call it unfair. It is brains vs. AI, after all, and that will always be true of brains no matter the task - fatigue is a factor, and this is where AI will have important advantages in the future.
- keldaris 10y ago
- eutectic 10y agoI wonder if it is possible to achieve super-human play purely using reinforcement learning to train deep neural networks.
- LAMike 10y agoWonder how long it will be until there is a crowdsourced AI winning the World Series of Poker?
- jobigoud 10y agoAre non-human players allowed to play?
- 36bydesignBL 10y agoReally long. It will still have to get lucky, or at least avoid being unlucky. This is why amateurs routinely crash the final table instead of just pros.
- source99 10y agoThese are heads up matches. The WSOP is 10 handed tables. I would guess the algorithms' skills may not work quite as well in 10 handed play.
- misja111 10y agoVery very long. First of all, this bot was capably only of analysing heads up play. At the WSOP there can be maximum 10 players at the table, and with each extra player, the number of combinations to analyse grows exponentially. Second, in the WSOP it is key to exploit weaker opponents. This bot was able to find almost perfect play against expert opponents but exploiting weak play is a different ballgame, especially if you are facing a mix of strong and weak players at a multi handed table.
- sagivo 10y agoI would love to know where they got their training data or if there's one publicly available.
- osti 10y agoThis is not neural net. It uses an algorithm called counterfactual regret minimization to compute the Nash equilibrium of the game, no data required.
- andrewprock 10y agoUnless things have changed significantly from the previous version (Claudico), this is not really no limit hold'em, as the stack sizes are reset every hand.
- poikniok 10y agoHow is that not no limit?
- notahacker 10y agoOne of the distinctive traits of no limit tournament is that on any given hand a player can be knocked out the game with a big enough bet, which affects how players are likely to play (as might one player amassing a large chip lead ove another if neither player opts to go all in on early hands) Of course it also makes games shorter and introduces a lot more variance, which isn't so good for assessing how well a computer plays.
- Jefro118 10y agoThis isn't a tournament though, it's more like playing a very long cash game where the stacks are reset on each hand. Whoever has the profit at the end of the cash game will be the winner.
- andrewprock 10y agoIn no limit, if you lose a hand you have fewer chips, which changes the applicable strategy. There is no version of no limit where every hand is played with the same number of chips. It is akin to only playing the first 10 moves of a chess game, then resetting.
- Jefro118 10y agoThis is indeed a version of no limit. What defines it as no limit is that there is "no limit" on the bet sizes. The fact that the chips are reset each hand doesn't mean it isn't no limit. The chess analogy would be more akin to resetting after the flop.
- skoutus 10y agoI wonder if the players tried colluding: coordinate their bets to fake a weakness so the AI start to adopt a poor strategy, then up the bets and stop the feign. I don't see how the AI can protect itself against that.
- erik998 10y agoIsn't coordination between players what really happens in most poker games...
- skoutus 10y agoThis is a multi-day competition. Players may coordinate in each round, but I mean to coordinate in each day. If you only coordinate over a short horizon then the complexity of your deception is lower. So for example, all players adopt a common feigned weakness on a given day, and let the computer believe those behaviors is part of a pattern to exploit. Then on the second day, up the bets and stop the feign. This happens in algorithmic trading, where traders would make a large number of low-valued, bad bets to mislead the algo. Then bet big and go the other direction.
- skoutus 10y agoI just looked it up and collusion is illegal in poker, though players could do it.
- hunl 10y agoThe aim of the AI isn't to adopt to poor strategies, rather to play an approximate optimal strategy itself. It's aiming to be unexploitable, the further the other players deviate from optimal, the more it wins. It's EV (expected value) comes from the other players not playing optimally, it doesn't care about exploiting individual weaknesses.
- skoutus 10y agoAI's aim is not to adopt poor strategies. It seeks to optimize, but the point is you can lead it to believe a point is global optimum, when in fact it is only local optimum. As to your point about the EV, this is why collusion can work. By colluding over a long enough horizon, the AI can believe that the average expected value to be something that it is not. If only one individual feign a weakness and the rest do not, then the strategy doesn't work.
- deleted 10y ago[deleted]
- nopinsight 10y agoI believe we are witnessing the Cambrian explosion of intelligences. The techniques behind Libratus (abstraction algorithm and game theory [1]) appear to be qualitatively distinct from those behind AlphaGo (deep reinforcement learning and MCMC) and DeepBlue (search and heuristics). An ecology of Artificial Intelligences, unbounded by our evolutionary history and neural architecture, could evolve to suit each particular task more effectively than our brains can. Promises and perils abound. [1] http://www.cs.cmu.edu/~sandholm/ http://www.cs.cmu.edu/~sandholm/ section "Algorithms and complexity of solving games"
- chii 10y ago> Promises and perils abound. what's the peril? I can only see promise ahead, if AI's progress continue unimpeded.
- FeepingCreature 10y agoCan I recommend Bostrom's Superintelligence: Paths, Dangers, Strategies?
- Certhas 10y agoCollapse of the current economic and political system as increased consumption can no longer compete with increased efficiencies. For once. But back when we were thinking about the perils of the internet we didn't correctly anticipate homogenization of media consumption and filter bubbles. People rather thought that it would be the other way around, everyone would have access to high quality information and enjoy diverse (long tail) media content. So we're likely to be wrong about the precise perils here, too.
- echelon 10y agoChess, Go, Poker... All feel like variations on the same theme. While it's obvious there is innovation being done, I want to see something more challenging. Something with more dimensionality and integration of several types of input. How about a machine that can beat someone at Smash Bros, a game with varied characters, complex comboing mechanics, and a nontrivial computer vision task? Or--more difficult by a few orders of magnitude--what about a robot that can beat someone at tennis? Or a team of robots that can best a professional basketball team? When do you suppose we'll begin to see these sorts of things? Within our lifetime, I hope?
- oculusthrift 10y agoIt's important to note that last time, the poker players beat CMU's AI by around 500k chips as well and they had the gall to declare it a "statistical tie". Yet if they win by chip they will claim to have "won".
- smohnot 10y agohttp://motherboard.vice.com/read/a-poker-playing-supercomputer-just-barely-lost-to-human-pros http://motherboard.vice.com/read/a-poker-playing-supercomput... Lost by $732,713, with $170 million bet... pretty close. But you're right, a victory by a chip will be declared a win
- oculusthrift 10y agoalso important to note they made the players play thousands of hands over weeks and many said how they played in a way to shorten the time they had to play. thereby not necessarily showing their "true" skill.
- patrick_haply 10y ago> also important to note they made the players play thousands of hands over weeks To your point, I wonder how they account for mental and physical fatigue. To a computer it makes no difference to play thousands of hands over such a long period of time or hundreds of hands over the course of a single day. Humans on the other hand don't have the same attention span as a computer.
- oculusthrift 10y agonot to mention that people play differently with fake money than real money.
- thefalcon 10y agoOn the Twitch Stream, Jason Les "promises" that "even though this is not real money" it really matters to him.
- mrkgnao 10y agoHow would someone with no knowledge of the game learn to play poker? Any good books that stress the mathematical/probabilistic(/game theoretic...?) side of things?
- davidivadavid 10y agoAssuming you at least know the rules of poker, you can probably start with the MIT course on poker theory, available here: https://www.youtube.com/watch?v=OTkq4OsG_Yc&list=PLUl4u3cNGP61kfOW3zAIfpNhf0piao8oo https://www.youtube.com/watch?v=OTkq4OsG_Yc&list=PLUl4u3cNGP...
- otoburb 10y agoPoker enthusiasts hang out at the TwoPlusTwo forums[1]. After reading up on combinatorics and learning the rules for numerous poker variants, diving into the forum posts will give you better insight into how players view the game, the situations they get into, and how they learn to analyze and read hands and other players. [1] http://www.twoplustwo.com/ http://www.twoplustwo.com/
- kriro 10y agoTrying to play GTO is an interesting (and very profitable) approach but ultimately it's not the most profitable approach. It makes sense for the AI to be constructed that way because the matches it's likely to play are against good opponents. Most profit in poker comes from playing against not so great opponents though. Against those you usually play extremely exploitable on purpose. That being said, the AI seems pretty impressive. Not sure how they picked the players I could think of a few HUNL players I'd rather see but they might not be interested in a 200k freeroll.
- xapata 10y agoIt's tough to decide what skill in poker really means. Is it that the AI can win against a good player or that the AI can earn money from bad players faster than other good players?
- angelofm 10y agoI played poker part time heads up (one against one) previously and the amount of study and analyses players have to do is huge to only get to a reasonable level. This challenge is very unfair to players so I wouldn't say it won, players have a massive disadvantage, every professional player has tracking software and a database to analyse every decisions that has been made. This is of course extremely important because you can model your strategy to exploit the suboptimal decisions made by your opponent, yet players have no access to any of these tools, so the bot adjusts it's play based on their human opponents but humans cannot do the same and are left with a guessing game. If they want to make a proper challenge then players need to have access to the tools they usually use playing in the Internet.
- Anderkent 10y ago>o the bot adjusts it's play based on their human opponents but humans cannot do the same and are left with a guessing game Do we know if it actually does it? I imagine it's much simpler to build a bot that plays a balanced profitable strategy rather than one that tries to build a model of their opponent and exploit it.
- deong 10y agoIt's stated in the article. During a day's play, the AI suggests plays based on its current knowledge. At night, the day's events are fed into the system for it to learn on for the next day. I assume there are probably papers that specify what form of learning is taking place, but the article didn't go into that level of detail and I haven't tried to track it down.
- bossx 10y agoIs this really fair? Part of Poker is "reading" when other players are bluffing, it's a lot more challenging to read when a computer is...
- hmate9 10y agoNot really. Online poker is much more popular. Plus the computer doesn't have a camera either. You have to deduce if your opponent is bluffing through the plays he/it is making.
- iopq 10y agoFirst of all, that's not true. Most players play several tables at the same time against the same player, up to four heads-up. That's something like 600 hands an hour in HU. There's little heads-up play in live games, other than at the end of the tournament, and the stack sizes are completely different. These are not tournament players, they're heads-up specialists. They most assuredly play online the majority of the time. Second of all, the computer doesn't read the human players, why would it matter? It's all up to actual strategies at that point.
- EGreg 10y agoHow good are the BEST poker bots right now at holdem with more than two players?
- czzarr 10y agoThat is very unconvincing as 49k hands is really nothing and not enough to iron out variance unless the edge is really big (which doesn't seem to be the case). Any serious poker player will tell you that. They should play 1-10 million hands (depending on the edge) in order to get a decent idea of where this is going.
- vannevar 10y agoYes, I'm wondering if anyone is tracking the relative strength of each player's hands. In the end, the bot should be declared the "winner" only if its winnings were disproportionately high in relation to the strength of its hands.
- tansey 10y agoThe usual way they reduce variance in these man vs. machine poker showdowns is to do "pairs" play. You have two humans playing simultaneously in isolated locations. The decks for both humans are the same, but player 1 and 2 are swapped for one human. That way, the bot strategy has to play both hands. It does totally eliminate variance, but they also take that into account and correct for it when looking at final outcomes usually. Right now the bot is up by something like 800K over 60K (out of 120K) hands. If that rate continues, it will win by around 1.6M or 400K per human. The blinds are 50/100, so that would equate to roughly 33 millibets (thousandths of a big blind per hand). That isn't too far off from standard win rates in bot vs. bot tournaments [1]. I'd say it's likely that the results of this tournament will be a statistically significant win for the bot. [1] http://www.computerpokercompetition.org/downloads/competitions/2014/results/results_2pn_tbr.pdf http://www.computerpokercompetition.org/downloads/competitio...
- czzarr 10y agoThat trick doesn't change the variance at all if decks are the same. Typical winrates in human vs human are between 1-5ptbb/100 where 1ptbb = two big blinds. At 1ptbb the variance is pretty big and north of 1million hands are probably necessary to establish an edge, whereas at 5ptbb the variance is much smaller and 100k hands are usually enough to converge to the expected value
- deleted 10y ago[deleted]