4 ms·

Does anyone know if this opens the door to using these techniques for poker (given that they've now show success on games of imperfect information)? Thus far t

by jonahx 8y ago

Does anyone know if this opens the door to using these techniques for poker (given that they've now show success on games of imperfect information)? Thus far the solutions to poker have involved solving the game tree through raw computational power and clever methods of information collapsing: http://science.sciencemag.org/content/347/6218/145 http://science.sciencemag.org/content/347/6218/145 But it seems the techniques used here might be both far more efficient, as well as able to better exploit human players (as opposed to playing a pure game theoretically optimal strategy, which is what they do now).

b_tterc_p 8y ago
Probably not. The AI demonstrated today was shown to be exploited with a strategy that wasn’t especially devious. It’s good at core mechanics but not strategically clever. I would bet on the optimal strategy over this one.
tialaramex 8y ago
So, your link is about Cepheus. It's important to understand that Cepheus isn't AI, it's an (asymptotically close to) optimal strategy for Limit Heads Up ("Limit" means you don't need to choose bet sizes, they are fixed, which makes the problem much simpler), and since poker is a game of probabilities the strategy is probabilistic too. ie this is like when somebody explains Tic-Tac-Toe, there isn't anything interesting going on inside the machine, the insight was purely mathematical, that there are one or more optimal ways to play this game, and this is one of them. You can literally look at the strategy right now, you can do it while playing against a program that plays by this strategy. But you won't beat it by doing that, that's the point of this optimal strategy, the best you can do is play this or some other equivalent optimal strategy back, in a home game you'll just pass chips back and forth forever. In contrast Poker AI is a thing, and at _No Limit_ Heads Up the AI state of the art is clearly better than human, named Libratus - it is nothing like Cepheus, there is no fixed strategy. For "Full Ring", the game of Poker as you've probably seen it played, which has more than half a dozen independent players, AI would be very challenging, not least because if humans realise they're at a disadvantage it would be essentially impossible to prevent them from colluding with other humans to get an advantage, even to some extent unconsciously.
- jonahx 8y ago
  Yes, I understood that. I was essentially asking if the techniques used by DeepMind could be leveraged to create a much more powerful version of AI's like Libratus, or to become very strong at games too big to solve for GTO solutions, such as full ring. > For "Full Ring", the game of Poker as you've probably seen it played, which has more than half a dozen independent players, AI would be very challenging This has already been done. There was an AI called Sonia which played both HU and full ring at expert human level or beyond, and was not based on GTO solutions. It created a model of every player which updated in real time, and exploited them the way expert humans do. I'm just curious if DeepMind would be able to achieve similar or better results.
- darkmighty 8y ago
  > not least because if humans realise they're at a disadvantage it would be essentially impossible to prevent them from colluding with other humans to get an advantage, even to some extent unconsciously Indeed versus a single opponent, you can play an "optimal" (e.g. Nash equilibrium strategy) and basically win/solve the game. But in a multiplayer setting (in poker and similar games), you rely on your ability to predict opponents. In the former you could essentially assume your opponent is as smart as possible ('rational' in game theory language), and if it is not rational it will necessarily do worse. In the latter however, by assuming every player is rational (e.g. playing N.E. strategies) you lack the ability to exploit weak players which are decidedly suboptimal, predict their moves, and capitalize. Thus winning is conditioned as much as the knowledge of your opponents, their skill and style, as in your own power to play probabilistic 'optimal' moves. A good simple example is also Rock-Paper-Scissors: a N.E. strategy just plays randomly -- it clearly cannot lose even against the most skilled players every -- but it also cannot win versus weak, predictable opponents. So e.g. in a tournament setting it would always lose, depending on the tournament structure (in particular if tournament stages are not all pairwise elimination). I find it a fascinating contrast to traditional game theory, and sort of conventional view of games, that there are "good players" and "bad players" in the sense of their execution being superior in an absolute sense, independent of opponent. In reality success in a variety of games (and real world scenarios) is won by tailoring your strategy to a particular opponent, using information from outside the game, and specifically predicting his plays vs. utilizing universal strategies (again this is particularly relevant in non-pairwise tournaments, non-zero sum games, etc). In real life, almost always the "game" is non-zero sum, multiplayer, with non-rational players, etc., even in a sports setting. Roger Federer might play slack and save his body in the early stages of a tournament vs a known weaker player, and give it all vs his most fearsome opponents. And also finally to quote "The Art of Strategy" (introductory game theory classic), 'There's always a bigger game.' (in real life it's the ultimately the entire Universe, and no one is quite sure of the Rules :p )
porpoisely 8y ago
And to add that, what if deepmind could get a poker history of the players at the table to create a poker profile of each player it is playing against. Having a percentage of a player's likelihood of folding and bluffing could keep it an advantage over a purely objective game theory aspect of the game. Maybe a certain player is more likely to bluff 5 hours into a game based off of player history analysis. Going further, imagine if DeepMind could get access to every players history outside of poker ( social media, purchasing records, medical records, etc ) and create a behavioral profile of each player. Game theory, poker profile, behavior profile, etc could give it a serious advantage over human players in addition to its advantage of never getting tired, frustrated, etc.
- b_tterc_p 8y ago
  deepmind is not necessary for that and also probably wouldn’t work. If you have player history you can construct perfect models for their playing habits and even update it as time goes on like a multi armed bandit problem. This can be simple probability heuristics and an optimization function. The problem with it is that the machine becomes biased to past results. Top human players are good because they adapt. I would put my money on the humans
erikb 8y ago
I think the bigger point in the future will be not AI beating known gambling activities (gambling in the sense that you put in money and have a random outcome about whether you get money back, let's not have a discussion about poker being gambling or competitive sport). The bigger point will be AI creating gambling activities that are even less resistable than our existing options. There might be digital drugs.

by jonahx 8y ago

b_tterc_p 8y ago

Probably not. The AI demonstrated today was shown to be exploited with a strategy that wasn’t especially devious. It’s good at core mechanics but not strategically clever. I would bet on the optimal strategy over this one.

tialaramex 8y ago

So, your link is about Cepheus. It's important to understand that Cepheus isn't AI, it's an (asymptotically close to) optimal strategy for Limit Heads Up ("Limit" means you don't need to choose bet sizes, they are fixed, which makes the problem much simpler), and since poker is a game of probabilities the strategy is probabilistic too. ie this is like when somebody explains Tic-Tac-Toe, there isn't anything interesting going on inside the machine, the insight was purely mathematical, that there are one or more optimal ways to play this game, and this is one of them. You can literally look at the strategy right now, you can do it while playing against a program that plays by this strategy. But you won't beat it by doing that, that's the point of this optimal strategy, the best you can do is play this or some other equivalent optimal strategy back, in a home game you'll just pass chips back and forth forever. In contrast Poker AI is a thing, and at _No Limit_ Heads Up the AI state of the art is clearly better than human, named Libratus - it is nothing like Cepheus, there is no fixed strategy. For "Full Ring", the game of Poker as you've probably seen it played, which has more than half a dozen independent players, AI would be very challenging, not least because if humans realise they're at a disadvantage it would be essentially impossible to prevent them from colluding with other humans to get an advantage, even to some extent unconsciously.

jonahx 8y ago
Yes, I understood that. I was essentially asking if the techniques used by DeepMind could be leveraged to create a much more powerful version of AI's like Libratus, or to become very strong at games too big to solve for GTO solutions, such as full ring. > For "Full Ring", the game of Poker as you've probably seen it played, which has more than half a dozen independent players, AI would be very challenging This has already been done. There was an AI called Sonia which played both HU and full ring at expert human level or beyond, and was not based on GTO solutions. It created a model of every player which updated in real time, and exploited them the way expert humans do. I'm just curious if DeepMind would be able to achieve similar or better results.
darkmighty 8y ago
> not least because if humans realise they're at a disadvantage it would be essentially impossible to prevent them from colluding with other humans to get an advantage, even to some extent unconsciously Indeed versus a single opponent, you can play an "optimal" (e.g. Nash equilibrium strategy) and basically win/solve the game. But in a multiplayer setting (in poker and similar games), you rely on your ability to predict opponents. In the former you could essentially assume your opponent is as smart as possible ('rational' in game theory language), and if it is not rational it will necessarily do worse. In the latter however, by assuming every player is rational (e.g. playing N.E. strategies) you lack the ability to exploit weak players which are decidedly suboptimal, predict their moves, and capitalize. Thus winning is conditioned as much as the knowledge of your opponents, their skill and style, as in your own power to play probabilistic 'optimal' moves. A good simple example is also Rock-Paper-Scissors: a N.E. strategy just plays randomly -- it clearly cannot lose even against the most skilled players every -- but it also cannot win versus weak, predictable opponents. So e.g. in a tournament setting it would always lose, depending on the tournament structure (in particular if tournament stages are not all pairwise elimination). I find it a fascinating contrast to traditional game theory, and sort of conventional view of games, that there are "good players" and "bad players" in the sense of their execution being superior in an absolute sense, independent of opponent. In reality success in a variety of games (and real world scenarios) is won by tailoring your strategy to a particular opponent, using information from outside the game, and specifically predicting his plays vs. utilizing universal strategies (again this is particularly relevant in non-pairwise tournaments, non-zero sum games, etc). In real life, almost always the "game" is non-zero sum, multiplayer, with non-rational players, etc., even in a sports setting. Roger Federer might play slack and save his body in the early stages of a tournament vs a known weaker player, and give it all vs his most fearsome opponents. And also finally to quote "The Art of Strategy" (introductory game theory classic), 'There's always a bigger game.' (in real life it's the ultimately the entire Universe, and no one is quite sure of the Rules :p )

porpoisely 8y ago

And to add that, what if deepmind could get a poker history of the players at the table to create a poker profile of each player it is playing against. Having a percentage of a player's likelihood of folding and bluffing could keep it an advantage over a purely objective game theory aspect of the game. Maybe a certain player is more likely to bluff 5 hours into a game based off of player history analysis. Going further, imagine if DeepMind could get access to every players history outside of poker ( social media, purchasing records, medical records, etc ) and create a behavioral profile of each player. Game theory, poker profile, behavior profile, etc could give it a serious advantage over human players in addition to its advantage of never getting tired, frustrated, etc.

b_tterc_p 8y ago
deepmind is not necessary for that and also probably wouldn’t work. If you have player history you can construct perfect models for their playing habits and even update it as time goes on like a multi armed bandit problem. This can be simple probability heuristics and an optimization function. The problem with it is that the machine becomes biased to past results. Top human players are good because they adapt. I would put my money on the humans

erikb 8y ago

I think the bigger point in the future will be not AI beating known gambling activities (gambling in the sense that you put in money and have a random outcome about whether you get money back, let's not have a discussion about poker being gambling or competitive sport). The bigger point will be AI creating gambling activities that are even less resistable than our existing options. There might be digital drugs.