3 ms·

Poker in a sense, is the most realistic game in that it has multiplayer, hidden information and stochastic aspects. While chess and go have huge state spaces (a

by Cybiote 10y ago

Poker in a sense, is the most realistic game in that it has multiplayer, hidden information and stochastic aspects. While chess and go have huge state spaces (as does no limit poker), they are also fairly straight forward due to their 2 player perfect information nature. If say you took an algorithm key to alphago, something like UCT, and tried to apply it to poker, it would diverge from nash equilibrium. Nash Equilibrium by the way, is the sense in which the game has solutions. In poker, you're looking at your expected value per decision (good players also do this at a meta-level), you're trying to place a bound on the other players' holdings and trying to bluff at an optimal rate. A good poker bot has to be capable of all this, which has proved extremely difficult. This is why it has proven difficult until just about this year, for researchers to be able to clearly say that at least in the heads up case and given enough compute to train, bots are at expert human level. What might turn out interesting is the fact that though headsup is more difficult for humans and multiplayer is arguably the simpler, the reverse seems to be true for bots. The state space multiplies such that naive application of equilibrium play and tree search simply do not cut it.

krick 10y ago
I don't understand. Maybe there are some good resources to read, so I could see it better? Essentially, you are explaining why poker it hard. This is no surprise, and I already included it into my question to an extent. The real question is, why is it harder for computers than for humans? Both seem to be completely oblivious to what is truly going on due to imperfect information nature of poker, and both seem to be able to make simple estimations. Computers are better at multiplying fractions (needed to estimate probability) and computers are better at not-worrying (needed to bluff well). I'd assume it is humans who are at disadvantage, and your post doesn't explain (it seems) why is it the otherwise.
- spectrum1234 10y ago
  As said before because of hidden information the optimal strategy appears to diverge unless you have more (all) the information in the decision tree. Basically you can never make a decision with the specific hand you have in isolation without knowing what you'd do with every other hand (and same for your opponent). This is very different than chess/go! So while poker may even have a simpler game tree than chess/go you need to have more of it upfront to solve the game, and being "off" means much much greater error. Depth matters less than breadth. Said another way in chess/go you can brute force get near optimal for the current decision, then reset and do again for the next decision. But with poker you have to solve for ALL POSSIBLE decisions (hands) up front (and inclusive of all possible board situations), and THEN do AGAIN 3 more times (pre-flop flop turn river) plus the numerous times you are raised means this is much more than 4 times total. Example: Bot is out of position with AdAh on AsKsQc facing a big raise. In isolation its probably best to re-raise and simulations show this makes the most money. But you also need to account for the times you have AT (no flush draw) and need to save AT money. But this logic doesn't stop here. Typically on the river out of position you will check the majority of your range if your opponent has bet every street (this is one of the few "proven" strategies because you always want the pot smaller out of position on average). So, you literally need to have the entire game solved in order to know HOW TO GET TO THE RIVER in a balanced way with good and bad hands. And, getting to the river to play profitably likely means giving up profitability on every other street - this is the diverging from optimal that default algorithms will be wrong on unless they have the entire game tree in front of them.
  - krick 10y ago
    This again has nothing to do with question in hand. Yes, poker is imperfect information game, everybody knows that. Why does it make it easier for humans than for computers? Humans cannot do these computations in the head any better than computers, quite the contrary.
    - spectrum1234 10y ago
      Hard to explain well. Let me try another way. Thinking through this more right now, it seems like the default way of computers to solve this penalizes them. Every time they get more depth it hurts them unless you account for more breadth. So lets say the programers know this. This STILL probably doesn't help much because...well idk. What's interesting is Tensor Flow is supposed to be massively parallel (breadth) and abstract the software/algorithms to do this. But if you follow this logic the answer must be gradient decent algorithms also get stuck somewhere... This must have to do with hyper-sensitivity to really having the full decision tree before optimizing. The answer seems to be if computers are modeling this exactly as humans do, (using neural networks which then simplify this problem with heuristics), it just takes this much computer power to simulate 1 human brain optimizing for this. Alternatively if they model it in a more brute force way it must take even more computer power because this is proving to be less efficient.
- nabla9 10y ago
  This is very good question. We covered why poker is hard. Now, lets cover why it is hard for computers. Simply: building model of the opponent(s) is part of poker. In games like Chess and Go without hidden information you can just "play the game" best you can. Other players are not important. Optimal strategy against good player is optimal against bad player are the same. It's just playing the game. In poker this is not true. Bets reveal information about the player's strategy and his hand. Figuring the other player out is part of the game. Against bad player, optimal strategy is usually different than against good player. Playing against tight-agressive is different than playing against tight-passive. In multi-way poker it's important to know how strong is the player sitting left to you and change your game accordingly.
  - krick 10y ago
    That makes sense. But does it mean that if I don't have a real idea of what I'm doing (i.e. you cannot assume my moves are optimal from my point of view) your expertise is essentially useless against me?
    - nabla9 10y ago
      If someone plays crazy, the solution is to go back playing simple near optimal poker aka playing the cards, calculate expected values and bet accordingly.
- leakybit 10y ago
  Because a computer has to adapt to the opponents strategy or else the human can simply exploit the rule based algorithm that the computer is following. Most dedicated online players are already using an arsenal of software for probability, statistics, and player profiling while playing, so a computer would really not have an edge with the math since the human is running the same computations. A human can start poking around and explore methods to fool the computer.

by Cybiote 10y ago

krick 10y ago

I don't understand. Maybe there are some good resources to read, so I could see it better? Essentially, you are explaining why poker it hard. This is no surprise, and I already included it into my question to an extent. The real question is, why is it harder for computers than for humans? Both seem to be completely oblivious to what is truly going on due to imperfect information nature of poker, and both seem to be able to make simple estimations. Computers are better at multiplying fractions (needed to estimate probability) and computers are better at not-worrying (needed to bluff well). I'd assume it is humans who are at disadvantage, and your post doesn't explain (it seems) why is it the otherwise.

spectrum1234 10y ago
As said before because of hidden information the optimal strategy appears to diverge unless you have more (all) the information in the decision tree. Basically you can never make a decision with the specific hand you have in isolation without knowing what you'd do with every other hand (and same for your opponent). This is very different than chess/go! So while poker may even have a simpler game tree than chess/go you need to have more of it upfront to solve the game, and being "off" means much much greater error. Depth matters less than breadth. Said another way in chess/go you can brute force get near optimal for the current decision, then reset and do again for the next decision. But with poker you have to solve for ALL POSSIBLE decisions (hands) up front (and inclusive of all possible board situations), and THEN do AGAIN 3 more times (pre-flop flop turn river) plus the numerous times you are raised means this is much more than 4 times total. Example: Bot is out of position with AdAh on AsKsQc facing a big raise. In isolation its probably best to re-raise and simulations show this makes the most money. But you also need to account for the times you have AT (no flush draw) and need to save AT money. But this logic doesn't stop here. Typically on the river out of position you will check the majority of your range if your opponent has bet every street (this is one of the few "proven" strategies because you always want the pot smaller out of position on average). So, you literally need to have the entire game solved in order to know HOW TO GET TO THE RIVER in a balanced way with good and bad hands. And, getting to the river to play profitably likely means giving up profitability on every other street - this is the diverging from optimal that default algorithms will be wrong on unless they have the entire game tree in front of them.
- krick 10y ago
  This again has nothing to do with question in hand. Yes, poker is imperfect information game, everybody knows that. Why does it make it easier for humans than for computers? Humans cannot do these computations in the head any better than computers, quite the contrary.
  - spectrum1234 10y ago
    Hard to explain well. Let me try another way. Thinking through this more right now, it seems like the default way of computers to solve this penalizes them. Every time they get more depth it hurts them unless you account for more breadth. So lets say the programers know this. This STILL probably doesn't help much because...well idk. What's interesting is Tensor Flow is supposed to be massively parallel (breadth) and abstract the software/algorithms to do this. But if you follow this logic the answer must be gradient decent algorithms also get stuck somewhere... This must have to do with hyper-sensitivity to really having the full decision tree before optimizing. The answer seems to be if computers are modeling this exactly as humans do, (using neural networks which then simplify this problem with heuristics), it just takes this much computer power to simulate 1 human brain optimizing for this. Alternatively if they model it in a more brute force way it must take even more computer power because this is proving to be less efficient.
nabla9 10y ago
This is very good question. We covered why poker is hard. Now, lets cover why it is hard for computers. Simply: building model of the opponent(s) is part of poker. In games like Chess and Go without hidden information you can just "play the game" best you can. Other players are not important. Optimal strategy against good player is optimal against bad player are the same. It's just playing the game. In poker this is not true. Bets reveal information about the player's strategy and his hand. Figuring the other player out is part of the game. Against bad player, optimal strategy is usually different than against good player. Playing against tight-agressive is different than playing against tight-passive. In multi-way poker it's important to know how strong is the player sitting left to you and change your game accordingly.
- krick 10y ago
  That makes sense. But does it mean that if I don't have a real idea of what I'm doing (i.e. you cannot assume my moves are optimal from my point of view) your expertise is essentially useless against me?
  - nabla9 10y ago
    If someone plays crazy, the solution is to go back playing simple near optimal poker aka playing the cards, calculate expected values and bet accordingly.
leakybit 10y ago
Because a computer has to adapt to the opponents strategy or else the human can simply exploit the rule based algorithm that the computer is following. Most dedicated online players are already using an arsenal of software for probability, statistics, and player profiling while playing, so a computer would really not have an edge with the math since the human is running the same computations. A human can start poking around and explore methods to fool the computer.